I thought this was pretty cool:
I wanted to make something like this after seeing a similar visualization of the Minecraft source repository. Obviously it looks a lot better with multiple people working on a long running project, but my employer would probably not appreciate me posting images of their repository. So I used my repository from a little HTML5 Tower Defense game I started earlier this year.
About the animation
The Minecraft animation has been made using Gource (link below). This is a tool that can read changes from a source control repository and visualize them in the way you see here. There are many things you can change; things like hiding/showing filenames, file types, captions as well as adding a custom logo and even avatars for the users of the system.
The problem is, though, that the projects I currently work on use Team Foundation Server. It’s a good source control system, among other things, but it does not seem to be very well liked by the open source community. As such, while Gource supports visualization of Git, Mercurial, Bazaar and SVN repositories, it does not natively support TFS.
How to create your own
I have come up with a solution though! As Gource can also parse log files, I have made a small program that will read a TFS repository and export changes made to a log file. You can then use Gource to visualize this data however you like. I will provide a step-by-step plan to create an animation from a TFS repository here.
You will need:
- Visual Studio 2012 installed (this is needed to get the TFS client API dlls). The express edition also works.
- Gource to visualize the changes.
- FFMPEG to compress the video into a reasonable size and viewable format.
- My TFS export tool to export a TFS repository into a log file. I understand that most people have qualms when asked to start an executable from the internet, no matter how many times I tell you it is safe. However, considering the target audience, I am sure you can decompile the assembly and find out whether it contains malicious code (I prefer Telerik JustDecompile for this). The actual source code may be released at a later point.
- A bunch of free disk space to store the temporary files and the final animation.
Step one: export source control history using TFS Exporter
Open a command prompt and type the name of the export tool (TfsExporter.exe). This will tell you what parameters it needs to proceed. If you are already authenticated to the server (which will usually be the case if you have recently used Visual Studio to access it), you should be good to go. If not, you will be asked for credentials.
The first parameter you will need to find is the source control URL for your team project collection. If it is a project hosted on VisualStudio.com, like mine, the url will be something like https://[yourusername].visualstudio.com/DefaultCollection. Else you can find this inside Visual Studio, by going to Configure Team Projects. The URL should be something like https://[tfs_url]/[TeamProjectCollection].
The second parameter is the project name. For me, this was “$/Olaf Tower Defense”.
The third parameter and onward are specific paths in the repository you would like to export. You can use this to get a specific folder, or to exclude certain folders; for instance if you have a folder with libraries or deliverables, you may wish to not have those in your animation. Alternatively, you can filter them out with Gource later on. If you don’t specify anything here, everything will be exported.
There is also an optional parameter which changes how usernames are exported. If you wish to use this, you need to specify it as the first parameter and move the rest of them over.
Step two: visualize log data using Gource
Now that we have one or more log files, we can start visualizing. Gource supports an interactive mode where you can hover your mouse over items to see information, pause the animation and more. This is a great way to quickly preview your animation before you export it to a file, since there are many options to set and rendering an animation every time will take a lot of time.
The basic way to get going is to call “gource.exe LogFile.txt“. This will show your animation on screen with the default settings. Have a look a gource.exe -H or the Gource website for some command line options. I recommend using –save-config to save your settings once you have changed a few of them, and then using –load-config instead of all of your command line parameters. Editing a config file seems easier (and more reusable) to me.
For reference, this is the config file I used for the animation at the start of this post:
[display] output-framerate=30 viewport=1920x1080 [gource] date-format=%A %B %#d, %Y disable-auto-skip=false file-filter=.*dll.* file-idle-time=0 font-size=20 hide=progress,mouse key=false max-files=0 seconds-per-day=2 title=www.olafkeijsers.net user-image-dir=avatars user-scale=2 bloom-intensity=0.5 bloom-multiplier=0.5
A few of these settings are worth explaining:
- file-idle-time: this sets how long files will stay on the screen after they have last been changed. I prefer to keep them forever, so the animation will show the actual source tree instead of only the things that changed last.
- max-files: for the same reason, I don’t limit the amount of files that can be shown on screen.
- seconds-per-day: this controls the speed of the animation. If you have a lot of users, you may want to increase this to show a little more detail.
- user-image-dir: if you place images with the same names as your users (depends on which NameFormat you used in step 1), these will be used to display user pictures instead of generic puppets.
- bloom-intensity and bloom-multiplier: I toned these down a little because I found the default bloom to be too bright.
Once you have found the settings you want to use, use the following to export the animation to a file:
gource.exe –load-config my.config LogFile.txt -o TemporaryFile.ppm
Note that this file contains uncompressed frames and as such will get quite large (about 100 times as large as the final animation, depending on compression). Do not close the window while it is doing this, the window will close automatically when it is done.
Step three: compress using FFMPEG
FFMPEG has even more options than Gource. As explaining them all would probably mean I run out of disk space at my hosting company, I will instead provide the command I used to compress my animation. I am by no means an expert on FFMPEG so there may be better ways to do this; feel free to leave a comment below if you know of a better way:
ffmpeg.exe -r 30 -f image2pipe -vcodec ppm -i TemporaryFile.ppm -shortest -vcodec libx264 -b:v 12000k -pix_fmt yuv420p -r 30 SourceControlVisualization.avi
Things to note are the frame rate (-r 30) and the bit rate (-b:v 12000k). For best results, the frame rate should match the frame rate you used in Gource. The bit rate is up to you. I prefer to see no artifacts, so since disk space is rather cheap I tend to go with higher bit rates. Since this kind of animation does not compress as well as regular video, a high bit rate is needed to keep the picture clear. For HD video (1080p) I found that 12000 kbit/s is a good number.
That is it, you’re done! Enjoy your video. One final thing: I have found that Windows Media Player did not play this video very well on my laptop, possibly due to the high bit rate. I suggest using VLC media player or something similar.
While I think some useful information can come from these visualizations, for me they are mostly for fun. It is pretty interesting to visually see the impact that certain refactorings have made on the source tree, and to see that after a while the projects tend to stabilize, with only small changes made here and there.
I hope this post was useful to someone. If you liked it or have suggestions, I would appreciate a comment. If you upload your video and link it here, that would be even better!