| Age | Commit message (Collapse) | Author |
|---|
|
|
|
It turns out the GitHub `updated_at` field doesn't change when new
commits are pushed to the repository, only when the repository config
etc. changes.
In order to update the mirrors when any update happens in the
repository, we need to look at both of those date values to see if
they've been updated.
Take the most recent of `updated_at` or `pushed_at` and set it to
`(database::Repo).updated_at`. This allows us to refresh the repo when
either of those dates change, catching all GitHub repo updates.
|
|
If the default branch on GitHub changed, change the local mirror's HEAD
to match the new default.
Need to store the default branch in the database now so we can find out
whether it changed.
|
|
Use 'rayon' to parallelise the repository processing. Each repository is
processed in a thread in the default 'rayon' pool.
In order to get thread-safe access to the database, I followed some
advice from a Stack Overflow answer by VasiliNovikov
(https://stackoverflow.com/users/1091436/vasilinovikov):
https://stackoverflow.com/questions/62560396/how-to-use-sqlite-via-rusqlite-from-multiple-threads/62560397#62560397
VasiliNovikov recommended creating a database connection pool using
'r2d2_sqlite'. This way we don't have to share a database connection
between threads, but each thread can have its own connection.
This also means we can remove mutable requirements in a bunch of places
involving our `database::Db` type since we're no longer managing the
database connections directly.
|
|
Turns out I need to specify all the flag I want in the open call,
including the one to open the database for reading and writing. This
fixes the "Error code 21: Library used incorrectly" error I was getting
earlier.
|
|
Trying to get rid of async. This compiles, but fails with the following
runtime error:
Error code 21: Library used incorrectly
Need to investigate further.
|
|
|
|
|
|
Check the repository description that comes back from the GitHub API
against our cached description in the database. Only write the new
description if it changed so we can avoid writing to the file in that
case.
|
|
Forgot to add commas when I added the additional fields.
|
|
This is simpler, and means we don't have to check if the database file
exists and only initialise if it doesn't. Here, we can just run the code
and trust it will do the right thing in both cases.
|
|
Store the repository description so we can check whether the description
was updated and copy it in our mirror accordingly.
|
|
If we haven't encountered a repository yet, mirror it to the filesystem.
Change `From<github::Repo>` to `From<&github::Repo>` so we don't consume
the repo and can use it later in the repo list loop.
|
|
If the repo was updated since we last cached it, update the `updated_at`
column to the new value so we know that we fetched the latest.
|
|
I think I like this better than checking the `RowNotFound` error.
|
|
Find out if the latest copy is more recent than the cached value in our
database.
|
|
Get rid of the boxed errors to make matching on errors easier.
|
|
|
|
Also change `repo_insert()` to take only a single repo instead of a
slice of them.
|
|
|
|
Sqlx already returns an appropriate error if no row was found.
|
|
|
|
Set up the database with a table for repositories so we can cache when
they were last updated.
|
|
|
