| Age | Commit message (Collapse) | Author |
|---|
|
af83/6884-tomtom-matrix--handle-error-when-response-doesn,t-inclu
Handle API error(s) in Tomtom matrix. Fixes #6884
|
|
|
|
I previously tried to correct a circular dependency problem in
a057276129b1f62b811743db3b8f867a05241ed3, but that didn't fix it (it was
intermittent, and came back).
After some wrangling, I've now deduced with some confidence that the
problem comes from `RouteWayCostCalculator`, which used
`TomTom::Matrix::RemoteError`. From the way it looks, this seems to mess
up the Rails autoloader since `tom_tom.rb` will try to load the `Matrix`
class from the `TomTom.matrix` call above. Or something.
In an attempt to fix the circular dependency error for real this time,
move the error class to a completely separate module from `Matrix`, and
refer to this when we need to use the error class.
Refs #6884
|
|
We might not always get a nicely formatted JSON
`['error']['description']` response body. Sometimes, like for example
when you use an incorrect API key, even with an 'application/json'
content type, TomTom will respond with:
<h1>Developer Inactive</h1>
What?
In that case, the response has a 403 status. In addition to checking for
an error in the response, should also be checking for the HTTP status
code.
Log the status code in the exception to give us more information about
what went wrong.
Update our existing tests now that `#check_for_error_response` takes a
response object instead of a JSON string.
Refs #6884
|
|
Refs #6884
|
|
Occasionally, the following error would appear in our logs:
NoMethodError
RouteWayCostWorker/perform
Error message
NoMethodError: undefined method `each_with_index' for nil:NilClass
Stack trace (show Rails)
/app/lib/tom_tom/matrix.rb: 83:in `extract_costs_to_way_costs!'
/app/lib/tom_tom/matrix.rb: 23:in `matrix'
/app/lib/tom_tom.rb: 24:in `matrix'
/app/app/services/route_way_cost_calculator.rb: 8:in `calculate!'
/app/app/workers/route_way_cost_worker.rb: 12:in `perform'
…ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb: 167:in `execute_job'
…ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb: 139:in `block (5 levels) in process'
/var/lib/gems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq.rb: 36:in `block in <module:Sidekiq>'
…ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb: 135:in `block (4 levels) in process'
….0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/chain.rb: 128:in `block in invoke'
….0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/chain.rb: 130:in `block in invoke'
…-4.2.10/lib/sidekiq/middleware/server/active_record.rb: 6:in `call'
….0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/chain.rb: 130:in `block in invoke'
…idekiq-4.2.10/lib/sidekiq/middleware/server/logging.rb: 10:in `call'
….0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/chain.rb: 130:in `block in invoke'
…kiq-4.2.10/lib/sidekiq/middleware/server/retry_jobs.rb: 74:in `call'
….0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/chain.rb: 130:in `block in invoke'
….0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/chain.rb: 133:in `invoke'
…ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb: 134:in `block (3 levels) in process'
…/gems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/logging.rb: 32:in `with_context'
…ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb: 132:in `block (2 levels) in process'
…ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb: 183:in `stats'
…ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb: 131:in `block in process'
/var/lib/gems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq.rb: 35:in `block in <module:Sidekiq>'
…ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb: 126:in `process'
…ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb: 82:in `process_one'
…ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb: 70:in `run'
…lib/gems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/util.rb: 17:in `watchdog'
…lib/gems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/util.rb: 26:in `block in safe_thread'
My best guess is that this was caused by TomTom responding with an
error, which we weren't handling previously. In that case, the response
would be a JSON string, but include an `'error'` field instead of
`'matrix'` and `'summary'`. Thus, when we'd try to
`matrix_json['matrix']`, it would fail.
Add a new method that checks for errors before we try to parse
`WayCost`s. If a server error is detected, we log the message to the
Rails log and return an empty array.
Refs #6884
|
|
GTFS import (first step)
|
|
We use integer values for distance (kilometres) and time (minutes). If
the values aren't very big, like bus stops inside a city for example,
the integer conversion will put distance and time at `0`, which isn't
correct.
To continue to use our chosen measurement units while still displaying
something that makes sense to users, snap any values >0 and <=1 to `1`.
Refs #6404
|
|
af83/6222-route-way-costs--use-TomTom-matrix-API-instead-of-batch
Route way costs use tom tom matrix api instead of batch. Refs #6222
|
|
|
|
New uniqueness validation for ccblocks
|
|
Rails serialises `BigDecimal`s as JSON strings to prevent loss of
precision. The `latitude` and `longitude` columns in `StopArea` are
stored as `BigDecimal`s. The trouble is that TomTom's API requires the
latitude & longitude values to be JSON floats, not strings.
Make a new JSON serialiser that converts the `BigDecimal` coordinates to
float to allow the values to be correctly interpreted by the API.
Refs #6222
|
|
Using a set ended up not working out. I needed to be able to index into
the list in `#extract_costs_to_way_costs!`, and sets aren't indexable.
This is because they're supposed to be unordered, though modern Ruby
implements `Set` with `Hash` under the hood, which is ordered in Ruby.
I like the idea of having a data structure that automatically eliminates
duplicates, but it wasn't meant to be, because for the extraction to
`WayCost`s, I need an ordered list. Rather than create a new
`OrderedSet` type, I just went the simple route and used an Array,
eliminating the duplicates manually because I know when duplicates are
supposed to occur due to the nature of the data set.
Remove the `#eql?` and `#hash` methods from `TomTom::Matrix::Point`.
Because we're not longer using `Set`, these methods don't need to be
implemented.
Refs #6222
|
|
Change this code to get stop IDs based on the change in
be3e1effcdea87909a181c7e9b12cf6867b1839d. It should use these IDs to
construct new `WayCost`s with the combination of stop IDs from departure
and arrival stops.
This doesn't work currently because the method has code that tries to
index the `points` collection, but it's a `Set`, and sets don't support
indexing and aren't ordered, so this code errors. Need to change the
`Set` to something else that will work here.
Refs #6222
|
|
Rewrite this method to accept `TomTom::Matrix::Point`s instead of plain
`Geokit::LatLng` coordinates. Do this because this method needs to take
the result of `#points_from_way_costs` as input, and that method now
returns a `Set` of `Point`s.
Refs #6222
|
|
We need to persist stop IDs in order to properly construct `WayCost`
objects from the costs returned from the TomTom matrix API.
In order to persist stop IDs, my idea here is to group together a point
and its corresponding ID into a bucket. When we later
`#extract_costs_to_way_costs!`, we'll be able to grab the correct ID for
a given coordinate to create a `WayCost` from it.
Here, we create a new `TomTom::Matrix::Point` class that encapsulates a
coordinate and an ID, and build a `Set` of those. I needed an `#eql?`
and `#hash` method on `Point` as described in the `Set` documentation
(https://ruby-doc.org/stdlib-1.9.3/libdoc/set/rdoc/Set.html) in order to
properly maintain a unique set.
Refs #6222
|
|
A new component to the `TomTom` module that asks TomTom's Matrix API
endpoint
(https://developer.tomtom.com/online-routing/online-routing-documentation/matrix-routing)
to compute `WayCost`s. The matrix API will give us all costs between
each pair of coordinates. This will enable us to provide costs for any
combination of points in a journey pattern.
Given a list of `WayCost`s, it will send all points from those costs to
the matrix API and return a list of all non-zero `WayCost`s between all
pairs of coordinates.
`points_from_way_costs()` extracts unique coordinates from the
`WayCost`s.
`points_as_params()` builds a list of points in the format expected by
the matrix API.
The response from the matrix API is formatted as a two-dimensional array
consisting of rows and columns that pair each "origin" point with each
"destination" point. We loop through this matrix and construct new
`WayCost` objects for each pair of coordinates.
At the moment, I haven't figured out how I want to save `WayCost` IDs
when creating the new pairs. Leaving that for later.
Refs #6222
|
|
section status fron Chouette::JourneyPattern
|
|
confirmed_at. Refs #6141
|
|
|
|
The JavaScript validation doesn't pass if we have more than two decimal
places (I think because it uses a number field step value of 0.01).
Round the values to allow them to pass frontend validation.
Refs #6203
|
|
Because we need to pass a Ruby hash to Rabl instead of a JSON string,
get rid of our serialiser and instead turn it into a function that just
converts the distance & time units.
Fix a bug in the test that had the `'1-2'` key as a symbol instead of a
string which was caused by a copy-paste from JSON and not being thorough
enough in search-and-replace.
Refs #6203
|
|
This serialiser will take `Route#costs` and convert the distance and
time fields from meters to kilometres and seconds to minutes
respectively. We need this because the frontend uses kilometre and
minute units while the TomTom API gives us the others (and we store the
data we get from TomTom without treatment).
Unfortunately, due to the way that Rabl works, this doesn't quite work
just yet. The serializer returns a string, and Rabl just puts this
string into the JSON output instead of a real JSON hash. Looks like I'm
going to have to convert my serializer into a generic converter.
Refs #6203
|
|
af83/6095-route--calculate-distance-and-time-cost-between-stops
Calculate distance and time cost between Route stops. Refs #6095
|
|
We say `TomTom` is disabled when no API key is present. If this is the
case, the `after_save` callback that uses it shouldn't be executed.
I had to change my `API_KEY` constant to an instance variable to be able
to change it for testing.
Refs #6095
|
|
A JSON serialiser for a collection of `WayCost`s. This will be used to
store `WayCost`s for a `Chouette::Route` in the `Route#costs` JSON
field.
Refs #6095
|
|
A new class that converts a list of `StopArea`s to `WayCost`s. This will
be called from `Chouette::Route` to get `WayCost`s from its stops, which
will then be JSON serialised and stored in `Route#costs`.
Update `WayCost`:
* Remove comment about calculating the ID automatically. It actually
needs to be the same as the `JourneyPattern#cost` ID (`key`), which is
a string with the IDs of the departure and arrival stops.
* Make `#==` check that `id`s are the same, which necessitates making
`id` a reader.
Refs #6095
|
|
Extract some code from `#batch` to allow us to test the part that takes
distance and time values from the response JSON and put fill them into
`WayCost`s.
In order for the test to work, had to add an `#==` method to `WayCost`
and make `distance` and `time` accessors.
The JSON file fixture is a copy of a response from the TomTom `/batch`
API. The file is pretty big. I'm not sure if maybe I should condense it
for the sake of test performance.
Refs #6095
|
|
Rename to `#convert_way_costs` since this method now lives inside the
`TomTom::Batch` class, so that part of the name became redundant.
Refs #6095
|
|
This change enables us to call `TomTom.batch(...)` instead of
`TomTom.new.batch(...)` a couple commits ago. This is nice because the
initialisation was kind of unnecessary for users of the class.
Refs #6095
|
|
Separate the functionality a little better by moving the `/batch`
endpoint code into a new class. The goal here is also to lay the
foundation for being able to call `TomTom.batch()` instead of
`TomTom.new.batch()`.
Refs #6095
|
|
This reverts commit f28a4b2c5b348bc12b455aa0cd76a9513103aea7.
As stated in that commit, I'm going to use the `/batch` endpoint instead
of `/calculateRoute` because I know that's already working.
|
|
A new method to provide access to the `/calculateRoute` endpoint, which
calculates costs for a single route through a bunch of waypoints.
This doesn't work currently because the API requires that I send
`supportingPoints` in the body of the request. Since I'm not too clear
on what those are, I'm going to cut off development of this method here
and just use `#batch` instead for my purposes.
My idea was to use this endpoint instead of `/batch` to allow us to
calculate a single route instead of having the API do `/calculateRoute`
for each of our point pairs. This seemed like it would be more
efficient, and give us our distance and time costs between each waypoint
all in one go, but since I'm not clear on how to use this API and
whether it will give us the correct data, I'm going to stick with
`/batch`. I'll probably be reverting this code. Just committing it now
in case it becomes useful in the future.
Refs #6095
|
|
Provides an interface to communicate with the TomTom API. Currently
includes a method `#batch` to make a batch routing request
(https://developer.tomtom.com/online-routing/online-routing-documentation/batch-routing).
Left a bunch of development-related code in just to preserve my
in-progress stages. Originally I was told to use the matrix routing API,
but that turned out to not match what we wanted. Namely, matrix routing
would produce a table with every point routed with every other point. We
instead want routes of each segment in a line, in order (or, just the
diagonal of the matrix). Using the batch API allows us to get the routes
we need without doing unnecessary work.
Update the `WayCost` class to provide `departure` and `arrival` methods
as readers, and make `id` an optional parameter for now. (We still need
to figure out how we're dealing with ID.)
Refs #6095
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
We now have a ModelDecorator and an "instance" decorator, all in the
same file, with the same API.
|
|
Used it in Lines#index and Lines#show, probably broke everything else
|
|
As well as the `to_html` method
|
|
- Implement new API
- Migrate the LineDecorator
- ensure no change on the "lines/index" view
|
|
|
|
|
|
af83/5281-workbench-import-structural-bug-reading-zip
5281 workbench import structural bug reading zip
|
|
5006 wb import filter refs with foreign lines
|
|
|
