I can understand how you arrived at your current solution. Keeping the databases separate makes it clear which data belongs to each plugin, and keeps concerns separate.
Although it offers clarity through a simple organization, this approach has some fairly significant downsides:
Connecting to a different db for each plugin is not going to be face. The database server will have a run a query N times to query N plugins, since it has to run each query against a separate database for each plugin.
Referential integrity is not easily enforcable across several databases (if at all) so it's quite likely your data may become inconsistent.
reduced flexibility and "dynamism": creating a new database quite a heavy operation, and so adding a new plugin becomes quite a heavy operation.
Finally, dealing with schema changes will be difficult - if you have a plugin that hasn't been updated to the latest schema, then it cannot be used in this scheme since all databases are assumed to have the same structure.
I would suggest a hybrid approach: each plugin continues to maintain it's own database which can be downloaded, and loaded at runtime, but rather than keeping the application and plugins in separate databases, the plugin data and application ddata are copied to a composite database. The composite database can be built at startup, or when the set of plugins changes, or as a new plugin version becomes available. This is workable since you mention each plugin database is only read, and not updated. (Rebuilding the database can be done so that the application data in the composite database is preserved.)
When the application data and the data from the plugins are integrated into one database you avoid the problems above:
the database server executes just one query rather than one per plugin
referential integrity is enforceable since all the data is maintained in one database.
finally, and most importantly, in my view - managing schema changes becomes possible. If there are plugins that haven't implemented your latest schema changes, the merging process can adapt data stored using the old schema while it is copying the plugin data to the composite dataabase (which always uses the latet schema.) For example, while copying a plugin using the old schema, default values can be added for new columns, changes in structure can be accomodated, columns/rows can be deleted etc. The same "schema upgrade" code can also be given to plugin developers to allow them to upgrade their plugin schema.
The merge process could also build an "installed_plugins" table listing all the plugins that are copied to the composite database. The table contains metadata about each plugin such as date of plugin update, time added, unique plugin id etc.
So, what does the composite database look like:
- plugin data stored as separate tables, using a view to combine them, The view would be a union of a corresponding table from each plugin, plus the a column for the plugin id. This will work, but is not likely to be efficient.
- plugin data for a given table type are all stored together. Referential integrity is much easier to maintain and queries are going to be faster, since a common index can be used across all plugins.
My preference is for the second option. But with all plugin data stored together, how to know which plugin each row comes from? Again there are two options:
- Add an extra column to the table with the plugin id/name. The advantage is that it's fast. The disadvantage is that it changes the base schema.
- For each table, have a companion table that maps a row ID to the plugin ID where the data was sourced from. This will be a little slower to query, but keeps the schema of your core data tables free from changes.
For simplicity, my preference would be for 1, adding an ID column. Since the tables in the composite database are built by a script or program, either scheme is simple to implement, and so it's mostly about preference or performance needs or whether it's important that the merge database uses the same schema for the core data tables as the original application database.
I feel that merging the data together is the right approach, for less pain, easier maintainance, flexibility and greater performance. But if you still have strong motives for keeping the data in separate tables, then at least keep them in the same database, and use either table name prefix, or better, schema names to keep the plugin data namespaces separate.
Good luck, which ever way you choose!