The automated deployment should be applied first on our Redshift DEV instance so the database changes and ETLs can be tested before deploying the changes to production. Automatic DeploymentsĪnalysts used to have Redshift permissions to run DDL queries manually, these permissions should be removed and automatic DB deployments should be allowed instead. All this pipeline should be integrated into our current Jenkins pipeline. CI/CD pipelineĪfter validating the database changes, an immutable artifact should be created together with the application code so automated DB deployments can be consistent, repeatable and predictable as application code releases. In addition, everybody gets their own database instance so the same tests can also be triggered locally while developing to get feedback even faster. Test code changes in a clean instance before testing them on a real one to make sure if the deployment will work as intended.Check correct dependencies between tables and views (column names, views referencing table columns, etc.).Check all tables/views/functions are following the naming conventions set by the team and that they align with the requirements of the organization.Some of the things we want to automatically test are: That way, analysts can get immediate feedback on SQL code, just as they do with application code. ![]() Once database code is checked in, a series of automated tests are immediately triggered. ![]() We will need to keep track of which migrations have been applied to each database. New environments can be created from scratch by applying these scripts.Īll database changes are migrations, therefore, each code change needs a unique identification.Prevent deployments where the database is out of sync with the application.Every change to the database is stored, allowing easy audit if any problems occur.Some immediate benefits this will give us: ![]() Database code is version controlledĭatabase code changes should be tracked in the same version control system as application code. We came up with a list of requirements and goals for this to happen: 1. The application code’s CI/CD pipeline was already robust so we thought about integrating Redshift’s scripts into the same CI/CD process. As you can see, in addition to not having a proper CI/CD pipeline for the scripts, all table fixes are appended to the same file without an easy way of keeping track of changes.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |