• Skip to site navigation
  • Skip to content
  • Skip to sidebar
  • Skip to footer
  • Go to archive page
Shadowmaster’s Lair
  • Home
  • Projects
  • Articles
  • About
  • Contact
  • Blog
‹ Previous | Blog index | Next ›

irker-svnpoller: Subversion poller and mail filter application for irker

Tuesday, October 2, 2012

Just as irker’s adoption rate is increasing, I have just completed work on a very simple application for Subversion repositories — two applications, in fact.

irker-svnpoller is a very simple script that polls a single commit log (not data) from a Subversion repository and delivers notifications to any number of channels using an irkerd running on the same host. It mimics the CIA bots’ formatting, much like nenolod’s irker CIA proxy, from which I borrowed a small amount of code.

irker-svnpoller → irkerd

But exactly how is this supposed to be useful to anyone, you may be wondering right now? Well, irker-svnpoller is not really intended to be used stand-alone. A timed poller script that tracks the last notified revision could come in handy, but people could get impatient waiting for their commits to appear in their IRC channels minutes later. I am well familiarized with the defects, quirks, and virtues of my primary audience—the Battle for Wesnoth and Wesnoth-UMC-Dev projects—, and this approach would simply not scale well over time.

Enter the first companion script, svnmail-filter. It reads email message headers from stdin to determine a commit’s revision number and the pertinent repository to probe using irker-svnpoller. Configuration is mostly done through a ruleset file using the JSON format.

Of course, svnmail-filter is not that useful on its own either. The idea is that procmail or some other MDA should pipe incoming email headers through svnmail-filter — and preferably, only those from legitimate sources, such as subscribed commit mailing lists. This is actually simpler than it sounds, and it is more or less inspired by CIA.vc’s perpetually broken mail-based SVN poller.

MDA → svnmail-filter → irker-svnpoller → irkerd

Since no service in the pipeline other than irkerd runs persistently in the background, this should be significantly more fault-resilient than CIA.vc’s approach, which apparently required a poller service to listen and act upon local requests. The downside is that the host running irker-svnpoller may need to create many short-lived SVN repository connections for individual commits in a chain. In Wesnoth’s case, SVN commit chains are rare enough, but their size often goes around a dozen individual commits or so. Regardless, this shouldn’t be terribly concerning for a production server with a decent low-latency uplink, and the overhead on the repository provider should be rather small compared to pushing massive commit diffs across the network.

Right now, the Wesnoth and Wesnoth-UMC-Dev projects are using this service as a stopgap measure until their respective providers—Gna.org and SourceForge.net—allow installing a hook that can either talk directly to an irkerd server, or to an instance of the aforementioned CIA proxy using the CIA XML-RPC method.

I am not all that keen on other people using a piece of software I developed and tested within less than three days without any prior experience working with Python. There are also various problems inherent to any application depending upon Subversion and its incompetent network transport layer.

Nonetheless, I published a Git repository on GitHub including a small amount of documentation to get started:

  • shikadilord/irker-svnpoller on GitHub

I am open to possible improvements coming from people intending to use this on production servers. In particular, if someone out there works with a commit mailing list where revision numbers can’t be found in mail subjects it would be necessary to adapt svnmail-filter a little to handle that case. Perhaps it might even be possible to skip the irker-svnpoller step for mailing lists where the notification message structure is constant and well documented.

Posted in IRC, Miscellaneous, Projects, Software, Wesnoth, Wesnoth-UMC-Dev at 23:54 UTC | 3 comments
Trackbacks
Trackback URI
No trackbacks
Comments
Linear | Threaded
Posted on 2012-10-04 21:24 (#1)
Ben Reser said:
A better solution is to use svnpubsub and the irker bridge I wrote for it.

All of which can be found under the tools portion of the SVN code base:
http://svn.apache.org/repos/asf/subversion/trunk/tools/server-side/svnpubsub/
Reply
Posted on 2012-10-04 22:13 (#1.1)
shadowmaster said:
That seems to require access to change the SVN repository hooks, though? The whole point of the irker-svnpoller pipeline is that it can be used with an existing configuration using mailing lists or other mail-based notification method.
Reply
Posted on 2012-10-05 01:04 (#1.1.1)
Ben Reser said:
Correct it does. However, I think it's a lot more useful to get 3rd parties to install a svnpubsub hook than an irker hook. There's certainly some value in what you're doing as an intermediate solution (or long term one for hosts that don't want to provide a better way).
Reply
Add Comment
All fields are optional. Your email address won't be publicly displayed.
HTML-Tags will be converted to Entities.
Standard emoticons like :-) and ;-) are converted to images.
 
 
 
‹ June ’13 ›
Mo Tu We Th Fr Sa Su
          1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
  • Recent posts
  • Archives
  • RSS/XML RSS 1.0
  • RSS/XML RSS 2.0
  • Atom/XML Atom 1.0
  • RSS/XML Comments
Twitter: @shikadilord
  • They're all available from here as always: http://t.co/vKL1027LPl13 hours ago
  • Fixed all image refs in my Wesnoth Evolution articles to refer to a host that doesn't dislike hotlinking specific files for no reason.13 hours ago
  • Yeah, like I was going to take anyone's opinion into account anyway.13 hours ago
  • RT @Wesnoth: Battle for Wesnoth turns ten today! http://t.co/ht8eAWm2IZ18 hours ago
  • P.S. those of you who hang around in my IRC channel enjoy wasting my time, all the time, so your answers won't count.18 hours ago
  • Dear followers (who are mostly spambots anyway), do you think I should fix my Wesnoth Evolution articles? [y/n] http://t.co/jfvwoeS7DG18 hours ago
  • Welcome back, #Wesnoth 1.10 add-ons server.1 day ago
  • Goodbye #Wesnoth 1.10 add-ons server.1 day ago
Categories
  • XML Frogatto
  • XML Hardware
  • XML IRC
  • XML freenode
  • XML Miscellaneous
  • XML Personal
  • XML Projects
  • XML Rei 2 IRC Bot
  • XML Wesnoth RCX
  • XML Wesnoth-TC
  • XML Site updates
  • XML Software
  • XML Web browsers
  • XML Web design
  • XML phpBB
  • XML Wesnoth
  • XML After the Storm
  • XML Invasion from the Unknown
  • XML Wesnoth Evolution
  • XML Wesnoth-UMC-Dev
Projects
  • Wesnoth Add-ons
  • Wesnoth RCX
  • Wesnoth-TC
  • Frogatto levels
  • Rei 2 IRC Bot
  • Wesnoth-UMC-Dev Registry
  • Shikadibot 0314
  • irker-svnpoller & svnmail-filter
  • phpBB 3.0 Mods/Hacks
Articles
  • Wesnoth Evolution
Links
  • Battle for Wesnoth
  • Wesnoth-UMC-Dev
  • Frogatto & Friends
Contact • Site Information & Disclaimer

Copyright © 2006-2013 by Ignacio R. Morelle. All rights reserved.
Powered by Poison Ivy/Dorset6 D18 and Serendipity.
Hosting by rewound.net and NearlyFreeSpeech.NET.

Serendipity PHP Weblog Valid XHTML 1.0 Transitional