Mining the Fedora Infrastructure Bus


If you listen to the Fedora Infrastructure Bus, you will hear things like:

fedora-infrastructure:org.fedoraproject.prod.fas.user.create
{u'certificate':
u'LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVSVENDQTY2Z0F3SUJ
BZ0lCSmpBTkJna3Fo\na2lHOXcwQkFRVUZBRENCb0RFTE1Ba0dBMVV
FQmhNQ1ZWTXgKQ3pBSkJnTlZCQWdUQWs1RE1SQXdE\nZ1lEVlFRSEV3ZFNZV3hs
YVdkb01SY3dGUVlEVlFRS0V3NUdaV1J2Y21FZwpVSEp2YW1WamRE
RVBN\nQTBHQTFVRUN4TUdabVZrYlhObk1ROHdEUVlEVlFRREV3Wm1aV1J0YzJjeER6QU5
CZ05WCkJDa1RC\nbVpsWkcxelp6RW1NQ1FHQ1NxR1NJYjNEU
UVKQVJZWFlXUnRhVzVBWm1Wa2IzSmhjSEp2YW1WamRD\nNXYKY21jd0hoY05NVEl3TnpFMU1
qSXdNRFE1V2hjTk1qSXdOekV6TWpJd01EUTVXakNCMURFT,
u'i': 1, u'timestamp': 1353031870.694725, u'topic':
u'org.fedoraproject.prod.fas.user.create', u'signature':
u'AvVPjUn0qP+W9qYLDkPFDBdmQDq66NJd2TuPfZZYYbwEkCSOmKsbNnGyGyVhM0n4SoqlNWR0KG49
\nz3FSBgHO2Tytnj+ZltMmeg8GNwaf5ys+bAaiL81AOTeG
ALtPD/iYGiXB0vIjZ685IdPMlu9ZZZej\nauUcUnKuacpzcy17/NA=\n',
u'msg': {u'user': {u'username': u'odemia'}, u'agent': {u'username':
u'odemia'}}}

which in plain text means that a new FAS username was created which goes by the name ‘odemia’.

Or, for example:

fedora-infrastructure:org.fedoraproject.prod.bodhi.update.comment
{u'certificate': u'LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVTVENDQT
dLZ0F3SUJBZ0lCRnpBTkJna3Fo\na2lHOXcwQkFRVUZBRENCb0RFTE1Ba0dBMVVFQmhNQ1
ZWTXgKQ3pBSkJnTlZCQWdUQWs1RE1SQXdE\nZ1lEVlFRSEV3ZFNZV3hsYVdkb01SY3dGUVl
EVlFRS0V3NUdaV1J2Y21FZwpVSEp2YW1WamRERVBN\nQTBHQTFVRUN4TUdabVZrYlhObk1RO
HdEUVlEVlFRREV3Wm1aV1J0YzJjeER6QU5CZ05WCkJDa1RC\nbVpsWkcxelp6RW1NQ1FHQ1Nx
R1NJYjNEUUVKQVJZWFlXUnRhVzVBWm1Wa2IzSmhjSEp2YW1WamRD\nNXYKY21jd0hoY05NVEl
1Evcit2TFFZZWoxN1cwCi0tLS0tRU5EIENFUlRJRklDQVRF\nLS0tLS0K\n', u'i': 1, 
u'timestamp': 1353038721.092251, 
u'topic': u'org.fedoraproject.prod.bodhi.update.comment', u'signature': 
u'kfsHMnH0zHp6t11woX9uH34t3qDGOtzRHHgrIuUMaBlB8XbNfXl2lQOzhR5iQd3iUIMUh71Q0UPL\njjaIvRJWxL/yz75oiq
btBMYUabi3X0mOmz3dj0rzGeIJVrwVdz+u5ePeBhR6rTDAvrmknrw6lgnr\nyVG7eFk6mQUGXk+MM9o=\n', u'msg':
{u'comment': {u'group': u'proventesters', u'author': u'adamwill', u'text': u'mostly working okay here', 
u'karma': 1, u'anonymous': False, u'timestamp': 1353038719.0, u'update_title': u'anaconda-18.29-1.fc18'}, 
u'agent': u'adamwill'}}

which corresponds to adamwill’s comment on bodhi [1].

If you start storing these messages emitted, you can mine for some interesting statistics. (I am storing all the messages in a file ‘messages’ using a particular format. See the Python source code at the end)

For example, in the last 20 hours or so, 16 new users registered with FAS:

$ cat messages | grep 'fedora-infrastructure:org.fedoraproject.prod.fas.user.create' | wc -l
16

And 175 edits was made to the various wiki pages:

$ cat messages | grep fedora-infrastructure:org.fedoraproject.prod.wiki.article.edit | wc -l
175

You can also look for rawhide and branch compose tree messages:

$ cat messages | grep fedora-infrastructure:org.fedoraproject.prod.compose
fedora-infrastructure:org.fedoraproject.prod.compose.rawhide.start
fedora-infrastructure:org.fedoraproject.prod.compose.rawhide.mash.start
fedora-infrastructure:org.fedoraproject.prod.compose.branched.start
fedora-infrastructure:org.fedoraproject.prod.compose.branched.mash.start
fedora-infrastructure:org.fedoraproject.prod.compose.rawhide.mash.complete
fedora-infrastructure:org.fedoraproject.prod.compose.rawhide.rsync.start
fedora-infrastructure:org.fedoraproject.prod.compose.rawhide.rsync.complete
fedora-infrastructure:org.fedoraproject.prod.compose.rawhide.complete

And as you can imagine, the possibilities are well, endless. Check out the documentation here at [2]. The consumer script I used is at [3]. Thanks to threebean for his help.

[1] https://admin.fedoraproject.org/updates/FEDORA-2012-18105/anaconda-18.29-1.fc18?_csrf_token=482cef87c4c42a4f60ba3faf97bbe16acad4e935
[2] Fedmsg Documentation
[3] Naive Consumer Script

About these ads

One thought on “Mining the Fedora Infrastructure Bus

  1. Neat, Amit! I’ll point you to datanommer too. It’s a daemon that stuffs fedmsg messages in a SQL database of your choosing. It’ll be neat to see all the different things people do with the bus data. Thanks for this!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s