The typo3.org project

Additionally, please also check the status of the AOE media deployment tool (like discussed in the meeting, too). Could this replace the existing solution?

Sorry couldn't find an appropriate documentation. So here's something to start with:

In general the deployment follows the continuous delivery paradigm. That means that whenever we want to deliver sth. to the production server we've to follow a certain (quality) workflow to make sure we break nothing. For typo3.org this means that we've basically 3 steps. A "package" step, followed by an installation on a testsystem which can be followed by delivery to the production system. As with continuous integration, all these steps should be repeated very often (ideally with every change in the codebase), that's meant to make sure that the developer gets a fast feedback wether his code can be integrated with the entire system or not.

That said - he's what the single steps are doing (at the moment):

0) Backup

As prerequisite for all steps we've a backup of the production system. I'll refer to that one as productionbackup . That includes the filesystem and a database-dump of the production system (excluding TER and cache-tables/files). For us the backup is made every 6 hours and pushed to the "CI" system (srv112) automatically.

1) The package step:

In this step we take the codebase from our SCM and "compare" it to the productionbackup. Whatever has changes will end up within the packages. This includes added files, changed database structures and changed configuration. The exact sources for the various extensions and database tables are configured within the project.yml file.

The result of the packaging is a so called "artifact" with can be though of as a "delta" between production and the SCM. All following steps will only verify / test this artifact WITHOUT modifying it (with few exceptions).

2) The installation on a test system

Once the "package" step is ready we try to test whether this can be installed on a system which is similar to production. For this purpose we've a system called "latest" - once a new artifact is ready, we first reset the latest system with the productionbackup. Once this is done we can think of "latest" to behave identical to production and install the artifact. The installation itself consists of some pre- and post-processing steps which make sure that filesystem-permissions are fine, caches are cleared, indexes are filled etc and some "sync" actions to get the new structures and files in the right place.

If that worked without problems - we can check "latest" automatically and manually (e.g. run unittests) to verify that the artifact didn't break things.

3) The production installation

Once the artifact was verified, we'll take it and install it on production. The installation procedure is identical to what we did on latest - just that the "reset" step is left out of course.

----
A followup system has to support the following things:

• Be able to create backups of production and use these to reset other systems
• Be able to find changes which which should be deployed (file-wise and database-wise)
• Be able to apply specfic changes to specific systems
• Have a reasonable speed (build should happen < 30mins per step)
  ----

As side-products we gain some other things which might not be relevant in other setups. We're e.g. able to build "full" packages witin the first step (delta agains zero) and these can be used to setup development-boxes very quickly. We're also able to integrate a couple more steps between 1) and 3) to add further validation.

One drawback with the deployment-tool version we use atm. is that is depends on SVN and is not able to use git repositories. An upgrade could be possible but will take some time - especially due to the fact that we've to change things direcly on the running prod. system