git is a distributed version control software package that enables very powerful management of software development projects. But what if you are developing an energy system model, whose inputs are large numbers of Excel spreadsheets?
Although Git works best with text files, Excel spreadsheets are not beyond its capabilities. The standard setup of git means that *.xlsx files are viewed as binary files. Actually, xlsx files are a zipped archive of XML files, and so can be unzipped and compared relatively easily with the help of a script to parse the XML.
TIMES models consist of a folder containing a nested file structure of Excel spreadsheets. On model compilation, the spreadsheets are read using the VEDA front-end (a proprietary software package) into a locked Access database. But to perform effective version control of a TIMES model, it is necessary only to record the changes to the Excel spreadsheets themselves.
Below, I'll outline the best practice for working with git and Excel spreadsheet files.
But first, here some examples of what is possible. Here I'm using the TIMES_DEMO model. This TIMES model contains four main spreadsheets, one for each of four key sectors.
Here's a picture of one sheet in the |
|
The python script produces text output of the worksheet in the following format: |
================================= Sheet: COM_Attribs[ 13 , 6 ] ================================= A1: * values associate with commodities C3: ~COMEMI C4: CommName D4: OILDST E4: GASNAT F4: COAHAR C5: ELCCO2N D5: 70.0 E5: 49.0 F5: 94.0 C7: <more emittants here> C9: ~COMAGG C10: CommName D10: ELCCO2N E10: CH4 C11: GHG D11: 0.001 E11: 0.21 F11: <more commodities> C13: <more commodities> |
Okay, so I'll now make a change and enter the value 72 in cell D5, replacing the existing value of 70. Then I saved the spreadsheet. |
|
Then I type |
$ git diff diff --git a/VT_DEMOT_ELC_V5.xlsx b/VT_DEMOT_ELC_V5.xlsx index 77ee5f0..8982118 100644 --- a/VT_DEMOT_ELC_V5.xlsx +++ b/VT_DEMOT_ELC_V5.xlsx @@ -356,7 +356,7 @@ Sheet: COM_Attribs[ 13 , 6 ] E4: GASNAT F4: COAHAR C5: ELCCO2N - D5: 70.0 + D5: 72.0 E5: 49.0 F5: 94.0 C7: <more emittants here> |
Installation and setup
As TIMES works only on Windows systems, other operating system specific installs will not be covered, but the python script linked to later and some of the git setup routines are applicable to those using Unix and OS X systems too. Feel free to adapt these guidelines to your systems.
- If you're happy working at a command prompt (i.e. typing in text commands) download and install git for Windows. Otherwise, download and install github client for Windows and setup an account. The github website has a range of very useful help files to follow on how to setup git on your computer. These include entering a username and e-mail address and setting up password caching for signing in to the website.
- Next, we need to install Python. There are various distributions of Python out there, but for ease, I'd recommend using Enthought Canopy - which is a free (for academics) distribution of the Python programming language and a whole bunch of useful packages. Request an academic licence and then download and install.
- Now, download the Python script from my github repository online here and save the file as
git_diff_xlsx.py
in a memorable location on your drive. - Add the following lines to the global .gitconfig file - this is normally in your user folder e.g.
C:/Users/will/
:
[diff "zip"]
binary = True
textconv = python c:/path/to/git_diff_xlsx.py
Note that in the final line above - your need to include the path to thegit_diff_xlsx.py
file you downloaded in the previous step. - Add the following line to the repository's .gitattributes
*.xlsx diff=zip
That's setup over and ready to go. Now, when you navigate to a git repository and type the git diff
command, any changes to xlsx spreadsheet files will be recorded as human-readable differences.