<feed xmlns='http://www.w3.org/2005/Atom'>
<title>chouette-core/app/workers, branch 6132-forms-breadcrumb</title>
<subtitle>Chouette manage transport static data</subtitle>
<link rel='alternate' type='text/html' href='https://git.teddywing.com/fork/chouette-core/'/>
<entry>
<title>RouteWayCostWorker: Add comment about recursive workers</title>
<updated>2018-03-14T15:42:02+00:00</updated>
<author>
<name>Teddy Wing</name>
</author>
<published>2018-03-13T17:45:28+00:00</published>
<link rel='alternate' type='text/html' href='https://git.teddywing.com/fork/chouette-core/commit/?id=a5d070093a2561c4924a030338db9064cdff7a16'/>
<id>a5d070093a2561c4924a030338db9064cdff7a16</id>
<content type='text'>
Explain why we need to skip and un-skip the callback.

Refs #6095
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Explain why we need to skip and un-skip the callback.

Refs #6095
</pre>
</div>
</content>
</entry>
<entry>
<title>RouteWayCostWorker: Fix recursive worker call</title>
<updated>2018-03-14T15:42:02+00:00</updated>
<author>
<name>Teddy Wing</name>
</author>
<published>2018-03-13T17:10:46+00:00</published>
<link rel='alternate' type='text/html' href='https://git.teddywing.com/fork/chouette-core/commit/?id=13135a5251f2af3e85bb2bb1be9866477b7414e9'/>
<id>13135a5251f2af3e85bb2bb1be9866477b7414e9</id>
<content type='text'>
Since this method was run on `after_save` and in it
`RouteWayCostCalculator` calls `#update` on the given route, it caused
an infinite recursive call to the worker. To prevent this, wrap the
`#calculate!` call in methods that unset and reset the callback.

Refs #6095
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Since this method was run on `after_save` and in it
`RouteWayCostCalculator` calls `#update` on the given route, it caused
an infinite recursive call to the worker. To prevent this, wrap the
`#calculate!` call in methods that unset and reset the callback.

Refs #6095
</pre>
</div>
</content>
</entry>
<entry>
<title>RouteWayCostWorker: Switch to referential in order to find Route</title>
<updated>2018-03-14T15:42:02+00:00</updated>
<author>
<name>Teddy Wing</name>
</author>
<published>2018-03-13T17:08:16+00:00</published>
<link rel='alternate' type='text/html' href='https://git.teddywing.com/fork/chouette-core/commit/?id=3fe9aad1d003814c2f5a5de834ce42a028df1753'/>
<id>3fe9aad1d003814c2f5a5de834ce42a028df1753</id>
<content type='text'>
Just running `Chouette::Route.find(id)` doesn't give us anything if we
don't have a `Referential` selected. In order to properly get the
correct route, pass the referential ID to the worker and switch to that
referential before trying to select the route.

Refs #6095
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Just running `Chouette::Route.find(id)` doesn't give us anything if we
don't have a `Referential` selected. In order to properly get the
correct route, pass the referential ID to the worker and switch to that
referential before trying to select the route.

Refs #6095
</pre>
</div>
</content>
</entry>
<entry>
<title>Route: Add `#calculate_costs!` to populate `#costs` with `WayCost`s</title>
<updated>2018-03-14T15:42:02+00:00</updated>
<author>
<name>Teddy Wing</name>
</author>
<published>2018-03-13T14:37:55+00:00</published>
<link rel='alternate' type='text/html' href='https://git.teddywing.com/fork/chouette-core/commit/?id=dbcd099b3d38f7c584450edffc8bee0a2502b170'/>
<id>dbcd099b3d38f7c584450edffc8bee0a2502b170</id>
<content type='text'>
This new method will launch a worker that takes the route's `StopArea`s,
converts them to `WayCost`s, sends them to the TomTom API to calculate
distance and time costs, and saves those costs to the route's `costs`
field.

Refs #6095
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This new method will launch a worker that takes the route's `StopArea`s,
converts them to `WayCost`s, sends them to the TomTom API to calculate
distance and time costs, and saves those costs to the route's `costs`
field.

Refs #6095
</pre>
</div>
</content>
</entry>
<entry>
<title>Refs #6133; Full UX for exports</title>
<updated>2018-03-12T11:00:15+00:00</updated>
<author>
<name>Zog</name>
</author>
<published>2018-03-09T09:12:48+00:00</published>
<link rel='alternate' type='text/html' href='https://git.teddywing.com/fork/chouette-core/commit/?id=f65a9f30f600f092fe3596e9e4db3c015dd91616'/>
<id>f65a9f30f600f092fe3596e9e4db3c015dd91616</id>
<content type='text'>
Also added a *dumb* `WorkgroupExport` for testing purposes.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Also added a *dumb* `WorkgroupExport` for testing purposes.
</pre>
</div>
</content>
</entry>
<entry>
<title>Refs #6133: Async exports</title>
<updated>2018-03-12T11:00:15+00:00</updated>
<author>
<name>Zog</name>
</author>
<published>2018-03-08T14:58:34+00:00</published>
<link rel='alternate' type='text/html' href='https://git.teddywing.com/fork/chouette-core/commit/?id=9b1da47adecbfb8d5d148fce06ad1aece18b41d0'/>
<id>9b1da47adecbfb8d5d148fce06ad1aece18b41d0</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Refs #6133; Fix specs</title>
<updated>2018-03-12T11:00:13+00:00</updated>
<author>
<name>Zog</name>
</author>
<published>2018-03-07T06:50:02+00:00</published>
<link rel='alternate' type='text/html' href='https://git.teddywing.com/fork/chouette-core/commit/?id=b810deb08cadd56caeb76eb81cff363458bb79a3'/>
<id>b810deb08cadd56caeb76eb81cff363458bb79a3</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>WorkbenchImportWorker: Use `File.unlink` instead of `#unlink`</title>
<updated>2018-02-22T08:11:01+00:00</updated>
<author>
<name>Teddy Wing</name>
</author>
<published>2018-02-20T16:13:19+00:00</published>
<link rel='alternate' type='text/html' href='https://git.teddywing.com/fork/chouette-core/commit/?id=91d683f6ad7a818312e9dc37b88eccd916e3b4d0'/>
<id>91d683f6ad7a818312e9dc37b88eccd916e3b4d0</id>
<content type='text'>
I was getting this error when I removed the `rescue nil`s:

    Failures:

      1) WorkbenchImportWorker foreign lines uploads the following entries: ["OFFRE_TRANSDEV_20170301122517", "OFFRE_TRANSDEV_20170301122519"]
         Failure/Error: eg_file.unlink

         NoMethodError:
           undefined method `unlink' for #&lt;File:0x007fef1adbace8&gt;
         # ./app/workers/workbench_import_worker.rb:81:in `upload_entry_group_stream'
         # ./app/workers/workbench_import_worker.rb:52:in `upload_entry_group'
         # ./app/services/zip_service.rb:61:in `&lt;&lt;'
         # ./app/services/zip_service.rb:61:in `finish_current_output'
         # ./app/services/zip_service.rb:35:in `add_entry'
         # ./app/services/zip_service.rb:27:in `block in _subdirs'
         # .../.gem/ruby/2.3.3/gems/rubyzip-1.2.1/lib/zip/entry_set.rb:38:in `block in each'
         # .../.gem/ruby/2.3.3/gems/rubyzip-1.2.1/lib/zip/entry_set.rb:37:in `each'
         # .../.gem/ruby/2.3.3/gems/rubyzip-1.2.1/lib/zip/entry_set.rb:37:in `each'
         # .../.gem/ruby/2.3.3/gems/rubyzip-1.2.1/lib/zip/central_directory.rb:182:in `each'
         # ./app/services/zip_service.rb:26:in `_subdirs'
         # .../.gem/ruby/2.3.3/gems/rubyzip-1.2.1/lib/zip/file.rb:133:in `open_buffer'
         # ./app/services/zip_service.rb:21:in `block in subdirs'
         # ./app/workers/workbench_import_worker.rb:39:in `each'
         # ./app/workers/workbench_import_worker.rb:39:in `each_with_index'
         # ./app/workers/workbench_import_worker.rb:39:in `upload'
         # ./app/workers/workbench_import_worker.rb:19:in `perform'
         # ./spec/workers/workbench_import_worker_spec.rb:68:in `block (4 levels) in &lt;top (required)&gt;'
         # ./spec/workers/workbench_import_worker_spec.rb:68:in `block (3 levels) in &lt;top (required)&gt;'
         # ./spec/workers/workbench_import_worker_spec.rb:13:in `instance_eval'
         # ./spec/workers/workbench_import_worker_spec.rb:13:in `block in expect_upload_with'
         # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/dependencies.rb:268:in `load'
         # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/dependencies.rb:268:in `block in load'
         # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/dependencies.rb:240:in `load_dependency'
         # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/dependencies.rb:268:in `load'
         # .../.gem/ruby/2.3.3/gems/spring-commands-rspec-1.0.4/lib/spring/commands/rspec.rb:18:in `call'
         # -e:1:in `&lt;main&gt;'

Johan said that in his tests on his local machine, the temp files
weren't getting deleted. When I try running the specs, the temp files do
get deleted, so not sure what was going on. Figure it may be because of
the `rescue nil`s, so I decided to remove those in the hopes of always
ensuring we close and unlink the temp files.

The trouble with doing that, however, was that it caused the error and
thus test failures like the example above. I managed to fix the problem
by using `File.unlink` instead of the instance method.

Hopefully this works to ensure that the temp files are always deleted.

Refs #4315
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
I was getting this error when I removed the `rescue nil`s:

    Failures:

      1) WorkbenchImportWorker foreign lines uploads the following entries: ["OFFRE_TRANSDEV_20170301122517", "OFFRE_TRANSDEV_20170301122519"]
         Failure/Error: eg_file.unlink

         NoMethodError:
           undefined method `unlink' for #&lt;File:0x007fef1adbace8&gt;
         # ./app/workers/workbench_import_worker.rb:81:in `upload_entry_group_stream'
         # ./app/workers/workbench_import_worker.rb:52:in `upload_entry_group'
         # ./app/services/zip_service.rb:61:in `&lt;&lt;'
         # ./app/services/zip_service.rb:61:in `finish_current_output'
         # ./app/services/zip_service.rb:35:in `add_entry'
         # ./app/services/zip_service.rb:27:in `block in _subdirs'
         # .../.gem/ruby/2.3.3/gems/rubyzip-1.2.1/lib/zip/entry_set.rb:38:in `block in each'
         # .../.gem/ruby/2.3.3/gems/rubyzip-1.2.1/lib/zip/entry_set.rb:37:in `each'
         # .../.gem/ruby/2.3.3/gems/rubyzip-1.2.1/lib/zip/entry_set.rb:37:in `each'
         # .../.gem/ruby/2.3.3/gems/rubyzip-1.2.1/lib/zip/central_directory.rb:182:in `each'
         # ./app/services/zip_service.rb:26:in `_subdirs'
         # .../.gem/ruby/2.3.3/gems/rubyzip-1.2.1/lib/zip/file.rb:133:in `open_buffer'
         # ./app/services/zip_service.rb:21:in `block in subdirs'
         # ./app/workers/workbench_import_worker.rb:39:in `each'
         # ./app/workers/workbench_import_worker.rb:39:in `each_with_index'
         # ./app/workers/workbench_import_worker.rb:39:in `upload'
         # ./app/workers/workbench_import_worker.rb:19:in `perform'
         # ./spec/workers/workbench_import_worker_spec.rb:68:in `block (4 levels) in &lt;top (required)&gt;'
         # ./spec/workers/workbench_import_worker_spec.rb:68:in `block (3 levels) in &lt;top (required)&gt;'
         # ./spec/workers/workbench_import_worker_spec.rb:13:in `instance_eval'
         # ./spec/workers/workbench_import_worker_spec.rb:13:in `block in expect_upload_with'
         # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/dependencies.rb:268:in `load'
         # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/dependencies.rb:268:in `block in load'
         # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/dependencies.rb:240:in `load_dependency'
         # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/dependencies.rb:268:in `load'
         # .../.gem/ruby/2.3.3/gems/spring-commands-rspec-1.0.4/lib/spring/commands/rspec.rb:18:in `call'
         # -e:1:in `&lt;main&gt;'

Johan said that in his tests on his local machine, the temp files
weren't getting deleted. When I try running the specs, the temp files do
get deleted, so not sure what was going on. Figure it may be because of
the `rescue nil`s, so I decided to remove those in the hopes of always
ensuring we close and unlink the temp files.

The trouble with doing that, however, was that it caused the error and
thus test failures like the example above. I managed to fix the problem
by using `File.unlink` instead of the instance method.

Hopefully this works to ensure that the temp files are always deleted.

Refs #4315
</pre>
</div>
</content>
</entry>
<entry>
<title>WorkbenchImportWorker: Reinstate creation of `./tmp/imports`</title>
<updated>2018-02-22T08:11:01+00:00</updated>
<author>
<name>Teddy Wing</name>
</author>
<published>2018-02-20T10:37:22+00:00</published>
<link rel='alternate' type='text/html' href='https://git.teddywing.com/fork/chouette-core/commit/?id=473d85d607605a4431c2deb32e38b026e64f15c6'/>
<id>473d85d607605a4431c2deb32e38b026e64f15c6</id>
<content type='text'>
If the directory doesn't exist, we should create it to ensure we don't
get errors trying to write to a location that doesn't exist. I had
removed this because I assumed the dir would be created during
deployment, but I guess I shouldn't assume. Also, we want to ensure the
dir gets created on dev machines without errors.

This line is updated from last time where we'd create
`Rails.root.join('tmp', 'imports')`. Now we use our `#temp_directory`,
which can be variable.

Refs #4315
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
If the directory doesn't exist, we should create it to ensure we don't
get errors trying to write to a location that doesn't exist. I had
removed this because I assumed the dir would be created during
deployment, but I guess I shouldn't assume. Also, we want to ensure the
dir gets created on dev machines without errors.

This line is updated from last time where we'd create
`Rails.root.join('tmp', 'imports')`. Now we use our `#temp_directory`,
which can be variable.

Refs #4315
</pre>
</div>
</content>
</entry>
<entry>
<title>WorkbenchImportWorker: Use temp files for entry group subdir Zips</title>
<updated>2018-02-22T08:11:01+00:00</updated>
<author>
<name>Teddy Wing</name>
</author>
<published>2018-02-12T16:38:00+00:00</published>
<link rel='alternate' type='text/html' href='https://git.teddywing.com/fork/chouette-core/commit/?id=67257e795fd4f11f5b34890f3114bd4ecdf80f7f'/>
<id>67257e795fd4f11f5b34890f3114bd4ecdf80f7f</id>
<content type='text'>
Instead of creating our own files for these subdirectory Zip files, use
temp files.

There are two advantages to doing this:

1. We can customise the location they get saved to. This means we can
   use `Rails.root` as a default, but specify a custom location in a
   Rails config variable.
2. The files will be automatically deleted on every garbage collection.

In actuality, #2 above was already handled by the `eg_file.unlink` line,
and #1 could be easily added to the existing code.

Using a `Tempfile`, though, makes the fact that these are temporary
files more explicit.

The reason why we want to be able to customise the location of these
tempfiles is that in our production environment set up by Puppet, the
`#{Rails.root}/tmp/imports` directory isn't writable by the application,
which causes this code to error. We thus want a way to change the path
in different environments.

We keep the `eg_file.unlink` call here because we want the temporary
file to be deleted as soon as possible. The Ruby documentation even
recommends it
(https://ruby-doc.org/stdlib-2.5.0/libdoc/tempfile/rdoc/Tempfile.html#class-Tempfile-label-Explicit+close):

&gt; it's good practice to do so: not explicitly deleting unused Tempfiles
&gt; can potentially leave behind large amounts of tempfiles on
&gt; the filesystem until they're garbage collected.

I got rid of the `#upload_entry_group_tmpfile` method in this change,
moving its contents to `#upload_entry_group_stream`. This was easy
enough to do since it was only called in that one place. The reason I
did this is because it took an open `File` object as an argument and
closed the file at the end of the method. That's a major side effect. I
don't think the method should be closing a file it doesn't own. After
moving this work to `#upload_entry_group_stream`, we can close and
unlink the file in the same place where it was created.

Refs #4315
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Instead of creating our own files for these subdirectory Zip files, use
temp files.

There are two advantages to doing this:

1. We can customise the location they get saved to. This means we can
   use `Rails.root` as a default, but specify a custom location in a
   Rails config variable.
2. The files will be automatically deleted on every garbage collection.

In actuality, #2 above was already handled by the `eg_file.unlink` line,
and #1 could be easily added to the existing code.

Using a `Tempfile`, though, makes the fact that these are temporary
files more explicit.

The reason why we want to be able to customise the location of these
tempfiles is that in our production environment set up by Puppet, the
`#{Rails.root}/tmp/imports` directory isn't writable by the application,
which causes this code to error. We thus want a way to change the path
in different environments.

We keep the `eg_file.unlink` call here because we want the temporary
file to be deleted as soon as possible. The Ruby documentation even
recommends it
(https://ruby-doc.org/stdlib-2.5.0/libdoc/tempfile/rdoc/Tempfile.html#class-Tempfile-label-Explicit+close):

&gt; it's good practice to do so: not explicitly deleting unused Tempfiles
&gt; can potentially leave behind large amounts of tempfiles on
&gt; the filesystem until they're garbage collected.

I got rid of the `#upload_entry_group_tmpfile` method in this change,
moving its contents to `#upload_entry_group_stream`. This was easy
enough to do since it was only called in that one place. The reason I
did this is because it took an open `File` object as an argument and
closed the file at the end of the method. That's a major side effect. I
don't think the method should be closing a file it doesn't own. After
moving this work to `#upload_entry_group_stream`, we can close and
unlink the file in the same place where it was created.

Refs #4315
</pre>
</div>
</content>
</entry>
</feed>
