Today we launched the beta of our [Ubuntu Upstream Report](https://edge.launchpad.net/ubuntu/+upstreamreport#). Jorge has [more details](https://stompbox.typepad.com/blog/2008/09/introducing-the.html) on how upstreams and Ubuntu contributors can make use of the report, but I wanted to spend a few moments telling the story behind the report.
Quite some time ago, we set ourselves up with the job of improving how we work with upstreams in a range of areas. To do this we hired Jorge to implement this work, and we decided to focus on bugs as our first action area. When trying to improve the relationship between two people, areas, or things, it is important to first define the *mechanics of interaction*. When thinking about the mechanics of interaction between Open Source projects, bugs offer a really exciting opportunity. Bugs are not only a common language between Open Source projects, but they also have a broadly similar schema – bug summary, description, state, associated patches, assignment etc. In a world with thousands of Open Source projects, bugs offer an interesting means of defining the connections between the thousands of dots. It is this kind of interconnected data that can be interesting in community improvement projects such as this.
Before we could begin improving our bug story though, we needed to understand more about the problem. Sure, we had ideas about how we could improve bug workflow, but the reality was that we were really just clutching at ideas and assumptions – we simply did not know enough about bug culture and the different approaches to bugs taken by upstream projects. Our first step was in understanding upstream bug workflow better. One key question here was:- exactly how can upstream projects make best use of our bugs?
To explain better, let me summarise the problem. Imagine you are using Ubuntu and something goes wrong. You suspect it is a bug, so you click *Help->Report a Problem* and you file a bug. We now have a bug in Launchpad in the Ubuntu project. Now, this bug is reasonably likely to actually exist in an upstream project. Imagine the bug was present in the GEdit text editor that we ship with Ubuntu – there is a distinct possibility that this bug is a bug in GEdit itself as opposed to a bug that we introduced while we build Ubuntu. It therefore makes sense to ensure that this distinction is noted, and to make this easier, we have a rather cool feature in Launchpad in which we can link an Ubuntu bug to a bug in an upstream bug tracker. In this particular example, if we found a bug in GEdit in Ubuntu and the bug is present in the upstream GEdit bug tracker, we can link the two bugs together. This link is also called a *watch*. This means that bug status changes in the upstream bug tracker will be reflected in the Ubuntu bug inside Launchpad. The goal here is that there should be one set of bug information that can be accessed from the upstream bug tracker and in Launchpad, and the two should be synced; multiple eyes on the same bug. Nice. 🙂
There were however two specific unknowns:
* Firstly, we had an suspicion that not that many Ubuntu bugs were getting linked to the upstream bug tracker. We had no fixed idea of this, but it was a hunch that we needed to quantify.
* Secondly, if we do link a bug upstream, we had no firm idea how useful an upstream actually find our bug data. Our discussions suggested very mixed reactions – a small project is likely to have a very different perspective on bugs than a large project. Just think about this in purely quantitative states – a small project will likely get fewer bugs, and these bugs can probably be dealt with by a small collection of volunteers. This is unlikely to scale to something like the Linux kernel or OpenOffice.org.
To help understand the latter problem more, we conducted an upstream survey, which some of you may have filled in. Surveys are a funny old beast, and a general issue survey can sometimes not yield particularly useful results, so we issued the same survey twice – first to an invitation-only range of upstreams who we knew would provide some objective commentary, and secondly we opened it up to anyone. A key element in this survey was asking about perspectives on bug workflow. We got a decent set of results (it is recognised in research that the magic number is 24 sets of results, so you don’t need hundreds of participants) and we assessed each set of results as well as combining the surveys. In addition to this we sat down and assessed other areas of bug workflow – watching how Ubuntu developers fix bugs and taking notes, developing bug jams further and improving our bug documentation.
But anyway, back to the upstream report specifically…
With every project that I approve on the community team at Canonical I want to see metrics. This is something we apply across the board. I have to admit, I am a bit of a graphing obsessive; it is interesting which conclusions and patterns that you can identify when you track a set of data, and this is particularly interesting when you match your data to your initiatives in the time-line. Of course, not everything can be graphed, but a lot can, and it really helps us build our community more effectively.
In terms of this project, I was keen to see graphs that show the number of upstream bug linkages going on, the total number of open vs. upstream bugs and how many bugs are fixed elsewhere. We could use these graphs to determine our progress in improving our bug workflow, but this was not enough – we also needed raw data about which projects needed the most focus. Which projects were struggling the most with bug figures? Which projects were not forwarding bugs upstream? Which projects didn’t have an upstream bug tracker registered in Launchpad? We had all the answers to these questions in Launchpad, but no means of gathering them. To fix this, we created the Ubuntu Upstream Report.
The Ubuntu Upstream Report that has hit beta today is the result of us sitting down and determining exactly what kind of data we wanted to know about upstream bug culture, and presenting this data in a means that helps us focus our efforts more effectively. The most critical focus with the report was to identify the Top 100 projects that need the most assistance with bug work – these projects are organised by open bug counts. We could then produce figures for each of these projects to identify how many upstream tasks are registered and how many of these tasks are links to upstream bugs. Combining these figures, it gives us an idea which projects are sucking at linking to upstream bugs.
With this kind of data available, it gives us the ability to drive our other initiatives that we have been building such as [Bug Jams](https://wiki.ubuntu.com/RunningBugJam) and [5-A-Day](https://wiki.ubuntu.com/5-A-Day). In fact, 5-A-Day is a key driver in how we can improve our bug linking story. Right now you can look at a project on the upstream report and click the number in the far right column to provide a list of bugs that have upstream tasks but no upstream link. A bug with an upstream task but no upstream link is likely to be considered an upstream bug, but it really needs the link to be made. All you need to do is find the upstream bug in the upstream bug tracker and link to it in the Ubuntu bug in Launchpad; this is a really useful and simple contribution to your 5-A-Day – nail five of these suckers a day, and you are flying. How cool is that – we can let our incredible 5-A-Day community know exactly what easily needs fixing in the Top 100 projects that need attention. That is *pure class in a glass*.
This is just one of a number of projects that we are working on to improve how we work with other entities in the Open Source world. We are really, really keen to not only build a strong and effective Ubuntu community, but to also ensure we can work as effectively with other projects and contributors too. Sure, we have some things to fix, but I am determined for us to resolve these problems and really drive through new opportunities to improve how we work together. This is one element in how we want to improve our relations with upstreams, so stay tuned for more as we develop our ideas.
