<feed xmlns='http://www.w3.org/2005/Atom'>
<title>chouette-core/spec/lib/tom_tom, branch master</title>
<subtitle>Chouette manage transport static data</subtitle>
<link rel='alternate' type='text/html' href='https://git.teddywing.com/fork/chouette-core/'/>
<entry>
<title>Move `TomTom::Matrix::RemoteError` to`TomTom::Errors::MatrixRemoteError`</title>
<updated>2018-05-04T10:54:25+00:00</updated>
<author>
<name>Teddy Wing</name>
</author>
<published>2018-05-04T10:49:06+00:00</published>
<link rel='alternate' type='text/html' href='https://git.teddywing.com/fork/chouette-core/commit/?id=9c442cd66c7d12266ba53e85949f775f6b842d2d'/>
<id>9c442cd66c7d12266ba53e85949f775f6b842d2d</id>
<content type='text'>
I previously tried to correct a circular dependency problem in
a057276129b1f62b811743db3b8f867a05241ed3, but that didn't fix it (it was
intermittent, and came back).

After some wrangling, I've now deduced with some confidence that the
problem comes from `RouteWayCostCalculator`, which used
`TomTom::Matrix::RemoteError`. From the way it looks, this seems to mess
up the Rails autoloader since `tom_tom.rb` will try to load the `Matrix`
class from the `TomTom.matrix` call above. Or something.

In an attempt to fix the circular dependency error for real this time,
move the error class to a completely separate module from `Matrix`, and
refer to this when we need to use the error class.

Refs #6884
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
I previously tried to correct a circular dependency problem in
a057276129b1f62b811743db3b8f867a05241ed3, but that didn't fix it (it was
intermittent, and came back).

After some wrangling, I've now deduced with some confidence that the
problem comes from `RouteWayCostCalculator`, which used
`TomTom::Matrix::RemoteError`. From the way it looks, this seems to mess
up the Rails autoloader since `tom_tom.rb` will try to load the `Matrix`
class from the `TomTom.matrix` call above. Or something.

In an attempt to fix the circular dependency error for real this time,
move the error class to a completely separate module from `Matrix`, and
refer to this when we need to use the error class.

Refs #6884
</pre>
</div>
</content>
</entry>
<entry>
<title>TomTom::Matrix#check_for_error_response: Handle HTTP error status codes</title>
<updated>2018-05-03T16:25:57+00:00</updated>
<author>
<name>Teddy Wing</name>
</author>
<published>2018-05-03T16:21:04+00:00</published>
<link rel='alternate' type='text/html' href='https://git.teddywing.com/fork/chouette-core/commit/?id=ad49ad52ee8e7cfbf4dc3f1bc34c533e186100b9'/>
<id>ad49ad52ee8e7cfbf4dc3f1bc34c533e186100b9</id>
<content type='text'>
We might not always get a nicely formatted JSON
`['error']['description']` response body. Sometimes, like for example
when you use an incorrect API key, even with an 'application/json'
content type, TomTom will respond with:

    &lt;h1&gt;Developer Inactive&lt;/h1&gt;

What?

In that case, the response has a 403 status. In addition to checking for
an error in the response, should also be checking for the HTTP status
code.

Log the status code in the exception to give us more information about
what went wrong.

Update our existing tests now that `#check_for_error_response` takes a
response object instead of a JSON string.

Refs #6884
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
We might not always get a nicely formatted JSON
`['error']['description']` response body. Sometimes, like for example
when you use an incorrect API key, even with an 'application/json'
content type, TomTom will respond with:

    &lt;h1&gt;Developer Inactive&lt;/h1&gt;

What?

In that case, the response has a 403 status. In addition to checking for
an error in the response, should also be checking for the HTTP status
code.

Log the status code in the exception to give us more information about
what went wrong.

Update our existing tests now that `#check_for_error_response` takes a
response object instead of a JSON string.

Refs #6884
</pre>
</div>
</content>
</entry>
<entry>
<title>TomTom::Matrix spec: Fix typo 'an' -&gt; 'a'</title>
<updated>2018-05-03T15:15:37+00:00</updated>
<author>
<name>Teddy Wing</name>
</author>
<published>2018-05-03T15:15:37+00:00</published>
<link rel='alternate' type='text/html' href='https://git.teddywing.com/fork/chouette-core/commit/?id=bfc4fab8943a28d2f30084a38d88678d7bfbbac4'/>
<id>bfc4fab8943a28d2f30084a38d88678d7bfbbac4</id>
<content type='text'>
Refs #6884
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Refs #6884
</pre>
</div>
</content>
</entry>
<entry>
<title>TomTom::Matrix: Handle error responses from TomTom</title>
<updated>2018-05-02T15:20:11+00:00</updated>
<author>
<name>Teddy Wing</name>
</author>
<published>2018-04-30T16:33:49+00:00</published>
<link rel='alternate' type='text/html' href='https://git.teddywing.com/fork/chouette-core/commit/?id=b6c0477552576e7f8575bf2dca6e7899b640c012'/>
<id>b6c0477552576e7f8575bf2dca6e7899b640c012</id>
<content type='text'>
Occasionally, the following error would appear in our logs:

    NoMethodError
    RouteWayCostWorker/perform

    Error message
    NoMethodError: undefined method `each_with_index' for nil:NilClass

    Stack trace (show Rails)
                                 /app/lib/tom_tom/matrix.rb:  83:in `extract_costs_to_way_costs!'
                                 /app/lib/tom_tom/matrix.rb:  23:in `matrix'
                                        /app/lib/tom_tom.rb:  24:in `matrix'
             /app/app/services/route_way_cost_calculator.rb:   8:in `calculate!'
                  /app/app/workers/route_way_cost_worker.rb:  12:in `perform'
    …ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb: 167:in `execute_job'
    …ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb: 139:in `block (5 levels) in process'
     /var/lib/gems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq.rb:  36:in `block in &lt;module:Sidekiq&gt;'
    …ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb: 135:in `block (4 levels) in process'
    ….0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/chain.rb: 128:in `block in invoke'
    ….0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/chain.rb: 130:in `block in invoke'
    …-4.2.10/lib/sidekiq/middleware/server/active_record.rb:   6:in `call'
    ….0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/chain.rb: 130:in `block in invoke'
    …idekiq-4.2.10/lib/sidekiq/middleware/server/logging.rb:  10:in `call'
    ….0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/chain.rb: 130:in `block in invoke'
    …kiq-4.2.10/lib/sidekiq/middleware/server/retry_jobs.rb:  74:in `call'
    ….0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/chain.rb: 130:in `block in invoke'
    ….0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/chain.rb: 133:in `invoke'
    …ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb: 134:in `block (3 levels) in process'
    …/gems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/logging.rb:  32:in `with_context'
    …ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb: 132:in `block (2 levels) in process'
    …ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb: 183:in `stats'
    …ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb: 131:in `block in process'
     /var/lib/gems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq.rb:  35:in `block in &lt;module:Sidekiq&gt;'
    …ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb: 126:in `process'
    …ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb:  82:in `process_one'
    …ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb:  70:in `run'
    …lib/gems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/util.rb:  17:in `watchdog'
    …lib/gems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/util.rb:  26:in `block in safe_thread'

My best guess is that this was caused by TomTom responding with an
error, which we weren't handling previously. In that case, the response
would be a JSON string, but include an `'error'` field instead of
`'matrix'` and `'summary'`. Thus, when we'd try to
`matrix_json['matrix']`, it would fail.

Add a new method that checks for errors before we try to parse
`WayCost`s. If a server error is detected, we log the message to the
Rails log and return an empty array.

Refs #6884
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Occasionally, the following error would appear in our logs:

    NoMethodError
    RouteWayCostWorker/perform

    Error message
    NoMethodError: undefined method `each_with_index' for nil:NilClass

    Stack trace (show Rails)
                                 /app/lib/tom_tom/matrix.rb:  83:in `extract_costs_to_way_costs!'
                                 /app/lib/tom_tom/matrix.rb:  23:in `matrix'
                                        /app/lib/tom_tom.rb:  24:in `matrix'
             /app/app/services/route_way_cost_calculator.rb:   8:in `calculate!'
                  /app/app/workers/route_way_cost_worker.rb:  12:in `perform'
    …ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb: 167:in `execute_job'
    …ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb: 139:in `block (5 levels) in process'
     /var/lib/gems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq.rb:  36:in `block in &lt;module:Sidekiq&gt;'
    …ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb: 135:in `block (4 levels) in process'
    ….0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/chain.rb: 128:in `block in invoke'
    ….0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/chain.rb: 130:in `block in invoke'
    …-4.2.10/lib/sidekiq/middleware/server/active_record.rb:   6:in `call'
    ….0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/chain.rb: 130:in `block in invoke'
    …idekiq-4.2.10/lib/sidekiq/middleware/server/logging.rb:  10:in `call'
    ….0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/chain.rb: 130:in `block in invoke'
    …kiq-4.2.10/lib/sidekiq/middleware/server/retry_jobs.rb:  74:in `call'
    ….0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/chain.rb: 130:in `block in invoke'
    ….0/gems/sidekiq-4.2.10/lib/sidekiq/middleware/chain.rb: 133:in `invoke'
    …ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb: 134:in `block (3 levels) in process'
    …/gems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/logging.rb:  32:in `with_context'
    …ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb: 132:in `block (2 levels) in process'
    …ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb: 183:in `stats'
    …ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb: 131:in `block in process'
     /var/lib/gems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq.rb:  35:in `block in &lt;module:Sidekiq&gt;'
    …ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb: 126:in `process'
    …ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb:  82:in `process_one'
    …ems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/processor.rb:  70:in `run'
    …lib/gems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/util.rb:  17:in `watchdog'
    …lib/gems/2.3.0/gems/sidekiq-4.2.10/lib/sidekiq/util.rb:  26:in `block in safe_thread'

My best guess is that this was caused by TomTom responding with an
error, which we weren't handling previously. In that case, the response
would be a JSON string, but include an `'error'` field instead of
`'matrix'` and `'summary'`. Thus, when we'd try to
`matrix_json['matrix']`, it would fail.

Add a new method that checks for errors before we try to parse
`WayCost`s. If a server error is detected, we log the message to the
Rails log and return an empty array.

Refs #6884
</pre>
</div>
</content>
</entry>
<entry>
<title>TomTom::Matrix: Serialize `BigDecimal` as float</title>
<updated>2018-03-27T14:46:18+00:00</updated>
<author>
<name>Teddy Wing</name>
</author>
<published>2018-03-27T13:24:27+00:00</published>
<link rel='alternate' type='text/html' href='https://git.teddywing.com/fork/chouette-core/commit/?id=c17665c0cc064c8a14af812dedd645977d110388'/>
<id>c17665c0cc064c8a14af812dedd645977d110388</id>
<content type='text'>
Rails serialises `BigDecimal`s as JSON strings to prevent loss of
precision. The `latitude` and `longitude` columns in `StopArea` are
stored as `BigDecimal`s. The trouble is that TomTom's API requires the
latitude &amp; longitude values to be JSON floats, not strings.

Make a new JSON serialiser that converts the `BigDecimal` coordinates to
float to allow the values to be correctly interpreted by the API.

Refs #6222
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Rails serialises `BigDecimal`s as JSON strings to prevent loss of
precision. The `latitude` and `longitude` columns in `StopArea` are
stored as `BigDecimal`s. The trouble is that TomTom's API requires the
latitude &amp; longitude values to be JSON floats, not strings.

Make a new JSON serialiser that converts the `BigDecimal` coordinates to
float to allow the values to be correctly interpreted by the API.

Refs #6222
</pre>
</div>
</content>
</entry>
<entry>
<title>TomTom::Matrix#points_from_way_costs: Use array instead of set</title>
<updated>2018-03-27T14:46:18+00:00</updated>
<author>
<name>Teddy Wing</name>
</author>
<published>2018-03-26T13:52:43+00:00</published>
<link rel='alternate' type='text/html' href='https://git.teddywing.com/fork/chouette-core/commit/?id=ebd46f068fbf11ea8793e9f7982b3a5291e21398'/>
<id>ebd46f068fbf11ea8793e9f7982b3a5291e21398</id>
<content type='text'>
Using a set ended up not working out. I needed to be able to index into
the list in `#extract_costs_to_way_costs!`, and sets aren't indexable.
This is because they're supposed to be unordered, though modern Ruby
implements `Set` with `Hash` under the hood, which is ordered in Ruby.

I like the idea of having a data structure that automatically eliminates
duplicates, but it wasn't meant to be, because for the extraction to
`WayCost`s, I need an ordered list. Rather than create a new
`OrderedSet` type, I just went the simple route and used an Array,
eliminating the duplicates manually because I know when duplicates are
supposed to occur due to the nature of the data set.

Remove the `#eql?` and `#hash` methods from `TomTom::Matrix::Point`.
Because we're not longer using `Set`, these methods don't need to be
implemented.

Refs #6222
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Using a set ended up not working out. I needed to be able to index into
the list in `#extract_costs_to_way_costs!`, and sets aren't indexable.
This is because they're supposed to be unordered, though modern Ruby
implements `Set` with `Hash` under the hood, which is ordered in Ruby.

I like the idea of having a data structure that automatically eliminates
duplicates, but it wasn't meant to be, because for the extraction to
`WayCost`s, I need an ordered list. Rather than create a new
`OrderedSet` type, I just went the simple route and used an Array,
eliminating the duplicates manually because I know when duplicates are
supposed to occur due to the nature of the data set.

Remove the `#eql?` and `#hash` methods from `TomTom::Matrix::Point`.
Because we're not longer using `Set`, these methods don't need to be
implemented.

Refs #6222
</pre>
</div>
</content>
</entry>
<entry>
<title>TomTom::Matrix#extract_costs_to_way_costs!: Try to include stop IDs</title>
<updated>2018-03-27T14:46:18+00:00</updated>
<author>
<name>Teddy Wing</name>
</author>
<published>2018-03-26T09:49:40+00:00</published>
<link rel='alternate' type='text/html' href='https://git.teddywing.com/fork/chouette-core/commit/?id=ddfc41f75cb52f02511e2acd6429270bc9c0ab19'/>
<id>ddfc41f75cb52f02511e2acd6429270bc9c0ab19</id>
<content type='text'>
Change this code to get stop IDs based on the change in
be3e1effcdea87909a181c7e9b12cf6867b1839d. It should use these IDs to
construct new `WayCost`s with the combination of stop IDs from departure
and arrival stops.

This doesn't work currently because the method has code that tries to
index the `points` collection, but it's a `Set`, and sets don't support
indexing and aren't ordered, so this code errors. Need to change the
`Set` to something else that will work here.

Refs #6222
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Change this code to get stop IDs based on the change in
be3e1effcdea87909a181c7e9b12cf6867b1839d. It should use these IDs to
construct new `WayCost`s with the combination of stop IDs from departure
and arrival stops.

This doesn't work currently because the method has code that tries to
index the `points` collection, but it's a `Set`, and sets don't support
indexing and aren't ordered, so this code errors. Need to change the
`Set` to something else that will work here.

Refs #6222
</pre>
</div>
</content>
</entry>
<entry>
<title>TomTom::Matrix#points_as_params: Use `TomTom::Matrix::Point`s</title>
<updated>2018-03-27T14:46:18+00:00</updated>
<author>
<name>Teddy Wing</name>
</author>
<published>2018-03-23T17:05:02+00:00</published>
<link rel='alternate' type='text/html' href='https://git.teddywing.com/fork/chouette-core/commit/?id=0a45ba811eefebb97bdc75f5e70da2be13186c28'/>
<id>0a45ba811eefebb97bdc75f5e70da2be13186c28</id>
<content type='text'>
Rewrite this method to accept `TomTom::Matrix::Point`s instead of plain
`Geokit::LatLng` coordinates. Do this because this method needs to take
the result of `#points_from_way_costs` as input, and that method now
returns a `Set` of `Point`s.

Refs #6222
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Rewrite this method to accept `TomTom::Matrix::Point`s instead of plain
`Geokit::LatLng` coordinates. Do this because this method needs to take
the result of `#points_from_way_costs` as input, and that method now
returns a `Set` of `Point`s.

Refs #6222
</pre>
</div>
</content>
</entry>
<entry>
<title>TomTom::Matrix#points_from_way_costs: Include stop IDs with points</title>
<updated>2018-03-27T14:46:18+00:00</updated>
<author>
<name>Teddy Wing</name>
</author>
<published>2018-03-23T16:01:16+00:00</published>
<link rel='alternate' type='text/html' href='https://git.teddywing.com/fork/chouette-core/commit/?id=e2cbcdb4dc32db05e2c6d7a7bb57952e1da6dab3'/>
<id>e2cbcdb4dc32db05e2c6d7a7bb57952e1da6dab3</id>
<content type='text'>
We need to persist stop IDs in order to properly construct `WayCost`
objects from the costs returned from the TomTom matrix API.

In order to persist stop IDs, my idea here is to group together a point
and its corresponding ID into a bucket. When we later
`#extract_costs_to_way_costs!`, we'll be able to grab the correct ID for
a given coordinate to create a `WayCost` from it.

Here, we create a new `TomTom::Matrix::Point` class that encapsulates a
coordinate and an ID, and build a `Set` of those. I needed an `#eql?`
and `#hash` method on `Point` as described in the `Set` documentation
(https://ruby-doc.org/stdlib-1.9.3/libdoc/set/rdoc/Set.html) in order to
properly maintain a unique set.

Refs #6222
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
We need to persist stop IDs in order to properly construct `WayCost`
objects from the costs returned from the TomTom matrix API.

In order to persist stop IDs, my idea here is to group together a point
and its corresponding ID into a bucket. When we later
`#extract_costs_to_way_costs!`, we'll be able to grab the correct ID for
a given coordinate to create a `WayCost` from it.

Here, we create a new `TomTom::Matrix::Point` class that encapsulates a
coordinate and an ID, and build a `Set` of those. I needed an `#eql?`
and `#hash` method on `Point` as described in the `Set` documentation
(https://ruby-doc.org/stdlib-1.9.3/libdoc/set/rdoc/Set.html) in order to
properly maintain a unique set.

Refs #6222
</pre>
</div>
</content>
</entry>
<entry>
<title>Add `TomTom::Matrix`</title>
<updated>2018-03-27T14:46:18+00:00</updated>
<author>
<name>Teddy Wing</name>
</author>
<published>2018-03-23T11:26:04+00:00</published>
<link rel='alternate' type='text/html' href='https://git.teddywing.com/fork/chouette-core/commit/?id=4bc33f14a148feb4056db1a2839303b07c67a6b3'/>
<id>4bc33f14a148feb4056db1a2839303b07c67a6b3</id>
<content type='text'>
A new component to the `TomTom` module that asks TomTom's Matrix API
endpoint
(https://developer.tomtom.com/online-routing/online-routing-documentation/matrix-routing)
to compute `WayCost`s. The matrix API will give us all costs between
each pair of coordinates. This will enable us to provide costs for any
combination of points in a journey pattern.

Given a list of `WayCost`s, it will send all points from those costs to
the matrix API and return a list of all non-zero `WayCost`s between all
pairs of coordinates.

`points_from_way_costs()` extracts unique coordinates from the
`WayCost`s.

`points_as_params()` builds a list of points in the format expected by
the matrix API.

The response from the matrix API is formatted as a two-dimensional array
consisting of rows and columns that pair each "origin" point with each
"destination" point. We loop through this matrix and construct new
`WayCost` objects for each pair of coordinates.

At the moment, I haven't figured out how I want to save `WayCost` IDs
when creating the new pairs. Leaving that for later.

Refs #6222
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
A new component to the `TomTom` module that asks TomTom's Matrix API
endpoint
(https://developer.tomtom.com/online-routing/online-routing-documentation/matrix-routing)
to compute `WayCost`s. The matrix API will give us all costs between
each pair of coordinates. This will enable us to provide costs for any
combination of points in a journey pattern.

Given a list of `WayCost`s, it will send all points from those costs to
the matrix API and return a list of all non-zero `WayCost`s between all
pairs of coordinates.

`points_from_way_costs()` extracts unique coordinates from the
`WayCost`s.

`points_as_params()` builds a list of points in the format expected by
the matrix API.

The response from the matrix API is formatted as a two-dimensional array
consisting of rows and columns that pair each "origin" point with each
"destination" point. We loop through this matrix and construct new
`WayCost` objects for each pair of coordinates.

At the moment, I haven't figured out how I want to save `WayCost` IDs
when creating the new pairs. Leaving that for later.

Refs #6222
</pre>
</div>
</content>
</entry>
</feed>
