Mighty morphin power elephant

Back in the 2013 I started playing with sqlalchemy to create a simple extractor from heterogeneous systems to be pushed in postgresql.
I decided to give the project a name which recalled the transformation and I called pg_chameleon.

To be honest I didn’t like sqlalchemy.  Like any other ORM adds an interface to the data layer with a mental approach to the data itself. I lost the interest to developing a migrator very soon, and after all there are thousands of similar tools thousands of times better than mine (e.g. the awesome pgloader)

However recently I revamped the project after discovering a python librarycapable to read the mysql replication protocol. In few weeks I cleaned all the sqlalchemy stuff, rebuilt the metadata extraction using the information_schema and finally I had an usable tool to replicate the data across the two systems.

I’ve also changed the license from GPL to the 2 clause BSD.

The tool requires testing. I’m absolutely sure is full of bugs and issues, but it seems to work quite nice.

Some key aspects:

  • Is developed in python 2.7. Bear with me, I’ll build a port to python 3.x when/if the project will get to an initial  release.
  • I use tabs (4 space tabs). Bear with me again. I tried to use spaces and I almost thrown my laptop out of the window
  • setup.py is not working. I’ll fix this as soon as I’ll do a release.
  • Yes, the sql part use the “hungarian notation” and the keywords are uppercase with strange indentation on the statements .  
  • The DDL are not yet replicated. I’m thinking to a clever approach to the problem.

That’s it. If you want to test it please do and try to break the tool :)

The tool is on github here: https://github.com/the4thdoctor/pg_chameleon/

Federico Campoli avatar
About Federico Campoli
Federico is a database administrator and an amateur python developer. He started his career as Oracle DBA in 2004 and fell in love with PostgreSQL in 2007.
comments powered by Disqus
Percona Live Europe 2018, Open Source Database Conference, Frankfurt 5-7 November 2018