Determining the effect of the Gorilla Toolkit archiving
How organisations who used the Gorilla Toolkit of Go libraries were able to understand the impact on their reliance on the projects when the maintainers declared they couldn't continue maintaining the project due to continued lack of support from the community.
(Note: adapted from the blog post Quantifying your reliance on Open Source software)
Context
In the Go ecosystem, the Gorilla Toolkit was a heavily used set of libraries for building web applications and RESTful APIs. Unfortunately in December last year, the maintainers announced that they were archiving the project due to a continued lack of support from the community:
We've archived the Gorilla Toolkit, a set of Go HTTP libraries & tools originally created back in Oct 2022.
— Matt Silverlock 🐀 (@elithrar) December 9, 2022
We never found a sticky maintainer and/or folks that were consistently contributing.
You can read more here: https://github.com/gorilla/.github/blob/687f52c9bcb80011a2935521c47498afa1551ece/profile/README.md
When the news broke, there was a lot of discussion around the Go community about whether it meant moving to different packages or looking at (a bit too late) forking and maintaining it.
Although most notably known for gorilla/mux
, the HTTP router, there are a number of other packages like gorilla/csrf
, gorilla/securecookie
and gorilla/sessions
that are much more risky when unmaintained, compared to the router which had not needed an update in over 2 years.
For projects and organisations that relied on the Gorilla Toolkit, understanding the impact was important.
Note that as of writing, it has been unarchived again, but the point still stands for this being a great case study.
Problem
To list:
- how many repositories directly depend on a Gorilla toolkit project
- how many repositories indirectly (transitively) depend on a Gorilla toolkit project
Additionally:
- which repositories directly depend on a Gorilla toolkit project
- which repositories indirectly (transitively) depend on a Gorilla toolkit project
Data
For instance, if we had the following data in our renovate
table:
platform | organisation | repo | package_name | version | current_version | package_manager | package_file_path | datasource | dep_types |
---|---|---|---|---|---|---|---|---|---|
github | dagger | dagger | github.com/gorilla/websocket | v1.5.0 | v1.5.0 | gomod | go.mod | go | ["require"] |
github | elastic | cloudbeat | github.com/gorilla/mux | v1.8.1 | v1.8.1 | gomod | go.mod | go | ["indirect"] |
github | elastic | cloudbeat | github.com/gorilla/websocket | v1.4.2 | v1.4.2 | gomod | go.mod | go | ["indirect"] |
github | golangci | golangci-lint | github.com/gorilla/websocket | v1.5.0 | v1.5.0 | gomod | tools/go.mod | go | ["indirect"] |
github | hashicorp | nomad | github.com/gorilla/mux | v1.8.0 | v1.8.0 | gomod | go.mod | go | ["indirect"] |
gitlab | gitlab-org | gitlab | github.com/gorilla/websocket | v1.5.1 | v1.5.1 | gomod | workhorse/go.mod | go | ["require"] |
gitlab | jamietanna | micropub-go | github.com/gorilla/context | v1.1.1 | v1.1.1 | gomod | go.mod | go | ["indirect"] |
gitlab | jamietanna | micropub-go | github.com/gorilla/securecookie | v1.1.1 | v1.1.1 | gomod | go.mod | go | ["indirect"] |
gitlab | jamietanna | micropub-go | github.com/gorilla/sessions | v1.1.3 | v1.1.3 | gomod | go.mod | go | ["indirect"] |
gitlab | jamietanna | oapi-codegen-example-project | github.com/gorilla/mux | v1.8.0 | v1.8.0 | gomod | go.mod | go | ["require"] |
gitlab | tanna.dev | dependency-management-data | github.com/gorilla/mux | v1.8.1 | v1.8.1 | gomod | go.mod | go | ["indirect"] |
gitlab | tanna.dev | dependency-management-data | github.com/gorilla/websocket | v1.5.0 | v1.5.0 | gomod | go.mod | go | ["indirect"] |
gitlab | tanna.dev | oidc-thumbprint | github.com/gorilla/schema | v1.2.0 | v1.2.0 | gomod | go.mod | go | ["indirect"] |
gitlab | tanna.dev | oidc-thumbprint | github.com/gorilla/securecookie | v1.1.1 | v1.1.1 | gomod | go.mod | go | ["indirect"] |
(Note: not all data listed above)
Query
As of dmd
v0.26.0, there was an inbuilt query that produced the following output:
$ dmd report gorillaToolkit --db dmd.db
Renovate
Direct dependencies
+------------------------------+---+
| PACKAGE | # |
+------------------------------+---+
| github.com/gorilla/mux | 4 |
| github.com/gorilla/websocket | 1 |
| github.com/gorilla/handlers | 1 |
+------------------------------+---+
Indirect dependencies
+---------------------------------+---+
| PACKAGE | # |
+---------------------------------+---+
| github.com/gorilla/websocket | 6 |
| github.com/gorilla/securecookie | 2 |
| github.com/gorilla/sessions | 1 |
| github.com/gorilla/schema | 1 |
| github.com/gorilla/mux | 1 |
| github.com/gorilla/context | 1 |
+---------------------------------+---+
This report has since been removed, since the project has been unarchived.
To perform a similar query, we could perform the following for direct dependencies:
select
package_name,
count(*)
from
renovate
where
package_name like 'github.com/gorilla/%'
and
-- instead of doing JSON querying with SQLite, simply do a string comparison
dep_types = '["require"]'
group by
package_name
order by
count(*) desc
And to look up indirect dependencies:
select
package_name,
count(*)
from
renovate
where
package_name like 'github.com/gorilla/%'
and
-- instead of doing JSON querying with SQLite, simply do a string comparison
dep_types = '["indirect"]'
group by
package_name
order by
count(*) desc
We can then also look up the repositories using the libraries with:
select
platform,
organisation,
repo,
arr.value as dep_type
from
renovate,
json_each(dep_types) as arr
where
package_name like 'github.com/gorilla/%'
group by
package_name
order by
dep_type desc
Which produces the following data, respectively:
package_name | count(*) |
---|---|
github.com/gorilla/websocket | 7 |
github.com/gorilla/mux | 7 |
github.com/gorilla/handlers | 3 |
github.com/gorilla/csrf | 1 |
And:
package_name | count(*) |
---|---|
github.com/gorilla/websocket | 13 |
github.com/gorilla/mux | 11 |
github.com/gorilla/css | 5 |
github.com/gorilla/securecookie | 3 |
github.com/gorilla/sessions | 1 |
github.com/gorilla/schema | 1 |
github.com/gorilla/handlers | 1 |
github.com/gorilla/context | 1 |
And:
platform | organisation | repo | dep_type |
---|---|---|---|
github | dagger | dagger | require |
github | deepmap | oapi-codegen | require |
github | elastic | beats | require |
github | tailscale | tailscale | require |
gitlab | jamietanna | micropub-go | indirect |
github | tailscale | tailscale | indirect |
gitlab | tanna.dev | oidc-thumbprint | indirect |
github | deepmap | oapi-codegen | indirect |
gitlab | jamietanna | micropub-go | indirect |