chouette-core/app/models/referential.rb, branch 5460_display_checksum

Merge branch 'master' into 5024-prevent-duplicate-referentials-from-being-created-during-parallel-db-transactions--rb201711271659

2018-01-03T10:57:42+00:00

Add log messages when Referential can't be created. Remove useless build_referenial invocation in ReferentialsController#create. Refs #5297

2017-12-28T11:16:19+00:00

BusinessCalendar => PurchaseWindow

2017-12-21T12:55:49+00:00

belongs_to referential

Referential#save_with_table_lock_timeout: Add argument

2017-12-13T17:30:58+00:00

I was getting test failures like this one:

    2) Workbenches show filtering filter by status should display archived referentials
       Failure/Error:
         def save_with_table_lock_timeout
           save_without_table_lock_timeout
         rescue ActiveRecord::StatementInvalid => e
           if e.message.include?('PG::LockNotAvailable')
             raise TableLockTimeoutError.new(e)
           else
             raise
           end

       ArgumentError:
         wrong number of arguments (given 1, expected 0)
       # ./app/models/referential.rb:64:in `save_with_table_lock_timeout'
       # .../.gem/ruby/2.3.3/gems/activerecord-4.2.8/lib/active_record/persistence.rb:241:in `update_attribute'
       # ./spec/features/workbenches_spec.rb:71:in `block (5 levels) in '
       # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/dependencies.rb:268:in `load'
       # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/dependencies.rb:268:in `block in load'
       # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/dependencies.rb:240:in `load_dependency'
       # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/dependencies.rb:268:in `load'
       # .../.gem/ruby/2.3.3/gems/spring-commands-rspec-1.0.4/lib/spring/commands/rspec.rb:18:in `call'
       # .../.gem/ruby/2.3.3/gems/spring-2.0.1/lib/spring/command_wrapper.rb:38:in `call'
       # .../.gem/ruby/2.3.3/gems/spring-2.0.1/lib/spring/application.rb:191:in `block in serve'
       # .../.gem/ruby/2.3.3/gems/spring-2.0.1/lib/spring/application.rb:161:in `fork'
       # .../.gem/ruby/2.3.3/gems/spring-2.0.1/lib/spring/application.rb:161:in `serve'
       # .../.gem/ruby/2.3.3/gems/spring-2.0.1/lib/spring/application.rb:131:in `block in run'
       # .../.gem/ruby/2.3.3/gems/spring-2.0.1/lib/spring/application.rb:125:in `loop'
       # .../.gem/ruby/2.3.3/gems/spring-2.0.1/lib/spring/application.rb:125:in `run'
       # .../.gem/ruby/2.3.3/gems/spring-2.0.1/lib/spring/application/boot.rb:19:in `'
       # -e:1:in `'

Sometimes an argument gets passed to the function and sometimes it
doesn't. Inspecting that argument in a debugger revealed what appeared
to be an options hash:

    (byebug) options
    {:validate=>false}

Add this optional argument to get past the test failures. At this point
I'm not even really considering whether this is a good idea or not, I
just want this to be done.

Refs #5024

Prevent error in Referential validation/creation when some attributs are defined (organisation, name, etc)

2017-12-12T20:26:37+00:00

Referential: Lock table on :update

2017-12-12T16:01:49+00:00

We had been locking the `referentials` table on :create, but we also
want to handle :update. Paired on this with Johan.

When I used:

    referential_2.metadatas << metadata_2

the referential was saved. To add the metadata without automatically
saving the referential + metadata, Johan suggested using the nested
attribute method:

    referential_2.metadatas_attributes = [metadata_2.attributes]

This allows us to add the metadata and still use the `#save` method to
lock the table.

Also change the callback from `before_validation` to `before_save`
because

    before_validation :lock_table, on: [:create, :update]

didn't work. That caused an error in our `expect`, as the `be_valid`
triggered the lock callback. To enable the callback on both :create and
:update, use a `before_save` instead.

Refs #5024

Referential: Raise an error if the table lock times out

2017-12-12T15:56:49+00:00

Paired with Johan on this one. There's an internal timeout on our table
lock. If it's reached, an
`ActiveRecord::StatementInvalid` error is raised.
Use a custom error instead by "overriding" `#save` with a method that
raises our custom error in that case instead. This will enable us to
provide a custom user-facing error in the event this happens.

Refs #5024

Referential#create_schema: Log benchmark time to create

2017-12-11T13:22:22+00:00

We want some measurements for how long it takes to create schemas to
give us some data to decide whether to put `#create_schema` in an
`after_commit` somehow (it didn't work the simple way). This at least
gives us a better idea of what's going on and how our table lock is
performing.

Refs #5024

Revert "Referential: Move `create_schema` to an `after_commit`"

2017-12-11T11:17:46+00:00

This reverts commit 5ae238936d3c91e70709c2ec4ed8a73a6f4524dc.

I was getting a bunch of test errors like this:

    1) Referentials destroy should remove referential
       Failure/Error: Apartment::Tenant.switch!(slug)

       Apartment::TenantNotFound:
         One of the following schema(s) is invalid: "test_1" "first", "shared_extensions"
       # .../.gem/ruby/2.3.3/gems/apartment-1.0.2/lib/apartment/adapters/postgresql_adapter.rb:92:in `rescue in connect_to_new'
       # .../.gem/ruby/2.3.3/gems/apartment-1.0.2/lib/apartment/adapters/postgresql_adapter.rb:85:in `connect_to_new'
       # .../.gem/ruby/2.3.3/gems/apartment-1.0.2/lib/apartment/adapters/abstract_adapter.rb:84:in `switch!'
       # ./app/models/referential.rb:139:in `switch'
       # ./app/controllers/referentials_controller.rb:26:in `show'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/abstract_controller/base.rb:198:in `process_action'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/action_controller/metal/rendering.rb:10:in `process_action'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/abstract_controller/callbacks.rb:20:in `block in process_action'
       # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/callbacks.rb:117:in `call'
       # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/callbacks.rb:555:in `block (2 levels) in compile'
       # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/callbacks.rb:505:in `call'
       # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/callbacks.rb:92:in `__run_callbacks__'
       # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/callbacks.rb:778:in `_run_process_action_callbacks'
       # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/callbacks.rb:81:in `run_callbacks'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/abstract_controller/callbacks.rb:19:in `process_action'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/action_controller/metal/rescue.rb:29:in `process_action'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/action_controller/metal/instrumentation.rb:32:in `block in process_action'
       # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/notifications.rb:164:in `block in instrument'
       # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
       # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/notifications.rb:164:in `instrument'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/action_controller/metal/instrumentation.rb:30:in `process_action'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/action_controller/metal/params_wrapper.rb:250:in `process_action'
       # .../.gem/ruby/2.3.3/gems/activerecord-4.2.8/lib/active_record/railties/controller_runtime.rb:18:in `process_action'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/abstract_controller/base.rb:137:in `process'
       # .../.gem/ruby/2.3.3/gems/actionview-4.2.8/lib/action_view/rendering.rb:30:in `process'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/action_controller/metal.rb:196:in `dispatch'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/action_controller/metal/rack_delegation.rb:13:in `dispatch'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/action_controller/metal.rb:237:in `block in action'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/action_dispatch/routing/route_set.rb:74:in `dispatch'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/action_dispatch/routing/route_set.rb:43:in `serve'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/action_dispatch/journey/router.rb:43:in `block in serve'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/action_dispatch/journey/router.rb:30:in `each'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/action_dispatch/journey/router.rb:30:in `serve'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/action_dispatch/routing/route_set.rb:817:in `call'
       # .../.gem/ruby/2.3.3/gems/devise_cas_authenticatable-1.10.0/lib/devise_cas_authenticatable/single_sign_out/rack.rb:14:in `call'
       # .../.gem/ruby/2.3.3/gems/warden-1.2.7/lib/warden/manager.rb:36:in `block in call'
       # .../.gem/ruby/2.3.3/gems/warden-1.2.7/lib/warden/manager.rb:35:in `catch'
       # .../.gem/ruby/2.3.3/gems/warden-1.2.7/lib/warden/manager.rb:35:in `call'
       # .../.gem/ruby/2.3.3/gems/rack-1.6.8/lib/rack/etag.rb:24:in `call'
       # .../.gem/ruby/2.3.3/gems/rack-1.6.8/lib/rack/conditionalget.rb:25:in `call'
       # .../.gem/ruby/2.3.3/gems/rack-1.6.8/lib/rack/head.rb:13:in `call'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/action_dispatch/middleware/params_parser.rb:27:in `call'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/action_dispatch/middleware/flash.rb:260:in `call'
       # .../.gem/ruby/2.3.3/gems/rack-1.6.8/lib/rack/session/abstract/id.rb:225:in `context'
       # .../.gem/ruby/2.3.3/gems/rack-1.6.8/lib/rack/session/abstract/id.rb:220:in `call'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/action_dispatch/middleware/cookies.rb:560:in `call'
       # .../.gem/ruby/2.3.3/gems/activerecord-4.2.8/lib/active_record/query_cache.rb:36:in `call'
       # .../.gem/ruby/2.3.3/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:653:in `call'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
       # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/callbacks.rb:88:in `__run_callbacks__'
       # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/callbacks.rb:778:in `_run_call_callbacks'
       # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/callbacks.rb:81:in `run_callbacks'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/action_dispatch/middleware/remote_ip.rb:78:in `call'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
       # .../.gem/ruby/2.3.3/gems/railties-4.2.8/lib/rails/rack/logger.rb:38:in `call_app'
       # .../.gem/ruby/2.3.3/gems/railties-4.2.8/lib/rails/rack/logger.rb:20:in `block in call'
       # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/tagged_logging.rb:68:in `block in tagged'
       # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/tagged_logging.rb:26:in `tagged'
       # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/tagged_logging.rb:68:in `tagged'
       # .../.gem/ruby/2.3.3/gems/railties-4.2.8/lib/rails/rack/logger.rb:20:in `call'
       # .../.gem/ruby/2.3.3/gems/request_store-1.3.2/lib/request_store/middleware.rb:9:in `call'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/action_dispatch/middleware/request_id.rb:21:in `call'
       # .../.gem/ruby/2.3.3/gems/rack-1.6.8/lib/rack/methodoverride.rb:22:in `call'
       # .../.gem/ruby/2.3.3/gems/rack-1.6.8/lib/rack/runtime.rb:18:in `call'
       # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/cache/strategy/local_cache_middleware.rb:28:in `call'
       # .../.gem/ruby/2.3.3/gems/rack-1.6.8/lib/rack/lock.rb:17:in `call'
       # .../.gem/ruby/2.3.3/gems/actionpack-4.2.8/lib/action_dispatch/middleware/static.rb:120:in `call'
       # .../.gem/ruby/2.3.3/gems/rack-1.6.8/lib/rack/sendfile.rb:113:in `call'
       # .../.gem/ruby/2.3.3/gems/railties-4.2.8/lib/rails/engine.rb:518:in `call'
       # .../.gem/ruby/2.3.3/gems/railties-4.2.8/lib/rails/application.rb:165:in `call'
       # .../.gem/ruby/2.3.3/gems/rack-1.6.8/lib/rack/urlmap.rb:66:in `block in call'
       # .../.gem/ruby/2.3.3/gems/rack-1.6.8/lib/rack/urlmap.rb:50:in `each'
       # .../.gem/ruby/2.3.3/gems/rack-1.6.8/lib/rack/urlmap.rb:50:in `call'
       # .../.gem/ruby/2.3.3/gems/rack-test-0.6.3/lib/rack/mock_session.rb:30:in `request'
       # .../.gem/ruby/2.3.3/gems/rack-test-0.6.3/lib/rack/test.rb:244:in `process_request'
       # .../.gem/ruby/2.3.3/gems/rack-test-0.6.3/lib/rack/test.rb:58:in `get'
       # .../.gem/ruby/2.3.3/gems/capybara-2.4.4/lib/capybara/rack_test/browser.rb:60:in `process'
       # .../.gem/ruby/2.3.3/gems/capybara-2.4.4/lib/capybara/rack_test/browser.rb:35:in `process_and_follow_redirects'
       # .../.gem/ruby/2.3.3/gems/capybara-2.4.4/lib/capybara/rack_test/browser.rb:21:in `visit'
       # .../.gem/ruby/2.3.3/gems/capybara-2.4.4/lib/capybara/rack_test/driver.rb:42:in `visit'
       # .../.gem/ruby/2.3.3/gems/capybara-2.4.4/lib/capybara/session.rb:227:in `visit'
       # .../.gem/ruby/2.3.3/gems/capybara-2.4.4/lib/capybara/dsl.rb:51:in `block (2 levels) in '
       # ./spec/features/referentials_spec.rb:193:in `block (3 levels) in '
       # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/dependencies.rb:268:in `load'
       # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/dependencies.rb:268:in `block in load'
       # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/dependencies.rb:240:in `load_dependency'
       # .../.gem/ruby/2.3.3/gems/activesupport-4.2.8/lib/active_support/dependencies.rb:268:in `load'
       # .../.gem/ruby/2.3.3/gems/spring-commands-rspec-1.0.4/lib/spring/commands/rspec.rb:18:in `call'
       # -e:1:in `'
       # ------------------
       # --- Caused by: ---
       # ActiveRecord::StatementInvalid:
       #   Could not find schema test_1
       #   .../.gem/ruby/2.3.3/gems/apartment-1.0.2/lib/apartment/adapters/postgresql_adapter.rb:86:in `connect_to_new'

Couldn't figure out how to get around this. The problem is that:

1. The referential is created
2. The app tries to switch to the newly-created referential's schema
3. The schema is created

But of course we can't switch to a schema that doesn't exist.

After a discussion with Luc & Alban, we decided to leave out the
`after_commit`. I'll be logging a benchmark of time it takes for schema
creation to give us an idea of the time & performance impact of the
schema creation and lock.

Merge remote-tracking branch 'origin/master' into 5024-prevent-duplicate-referentials-from-being-created-during-parallel-db-transactions--rb201711271659

2017-12-06T14:53:57+00:00