Design Decisions

This page is used to document some of the key design decisions that have been made by dependency-management-data.

All state is synced to the database

Once the database is prepared, you do not need to use any of the commands.

This ensures that there is no lock-in to using dependency-management-data's first-class tools, allowing you to take your data and use it with whichever tools you want to.

You will need to use the dmd CLI to prepare the database, after which you will be free to use the data as you wish.

This also links in with the use of SQLite as the database format, as it means that once prepared, there's only a single file that need be distributed and can be interacted with tools that users are comfortable with, rather than relying on them using the first-party commands.

The database is not as normalised as a well-designed database

The database is not as normalised as a database could be, and instead aims to be slightly easier to use (citation needed 🫣).

This is partially a purposeful decision to make sure that the data could be more easily queried, instead of relying on understanding of "to list all the dependencies in $repo, you need to query SELECT * FROM repos INNER JOIN renovate ON ...". This reduces the overhead of using dependency-management-data by slightly reducing the need for SQL knowledge, but does also make it a bit harder for users who do expect there to be a more normalised structure.

However, this is also a byproduct of not spending some up-front time thinking about the database structure, and may be improved in the future.

No foreign keys

dependency-management-data does not use Foreign Keys for the database.

This was investigated as part of Merge Request 66 but was ultimately dismissed, due to the requirement an opt-in for each SQLite client, which can be added for first-party commands, but third-party tooling may not always enable the foreign keys functionality in SQLite.

Additionally, it's not immediately straightforward to add foreign keys between unrelated tables. For instance, the owners table has a "foreign key" dependency on multiple tables, which would make describing this quite difficult.

However, although there is no foreign key constraint enforced, the ability to perform an INNER JOIN on these tables still exists, we just lose some of the referential integrity that foreign keys support.

SQLite only

SQLite is the only supported database backend.

SQLite is a convenient format for databases which has a great number of tools supporting it, and does not require a server running to interact with it.

As it is a database that is stored in a single file, this can make distribution of the file - across the organisation, between colleagues, or stored in a container image - very convenient.

The database should be treated as read-only and ephemeral

Once your data has been imported into dependency-management-data, and any additional steps like generating advisories, policy violations, and importing ownership information is performed, the database should be treated as read-only. Additionally, the database should be treated as ephemeral and re-created from scratch when new data needs importing.

This means that we do not consider database / table migrations or versioning of the data within the formats. This not only simplifies dmd's ability to need to worry about versioning, but also makes it much easier to reason about the database, knowing that once it's been "prepared", you're good to go.

This can, for instance, be generated centrally via a CI platform and then downloaded from a central artifact registry.

However, this does require that you align the version of the dmd CLI that is used to create the database and import data, and use the same version of dmd, dmd-web, or dmd-graph to make sure that the underlying database schemas are aligned.