Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
metrics [2014/08/13 14:59]
sanjeev [Recommended Approaches]
metrics [2015/07/19 15:30] (current)
jsmanrique [Points of Interest] solved issue to show Tools as a section
Line 5: Line 5:
 ==== Overview ==== ==== Overview ====
  
-=== "Not everything that can be counted, counts and not everything that counts can be counted"​. ​=== +//"Not everything that can be counted, counts and not everything that counts can be counted"​.//​ 
-//''​Albert Einstein''​// +**''​Albert Einstein''​** 
 +----
 Metrics have always played a key role in businesses and processes. Metrics have always played a key role in businesses and processes.
 But, in communities like software development,​ it is hard to quantify important aspects that contribute immensely. But, in communities like software development,​ it is hard to quantify important aspects that contribute immensely.
Line 19: Line 19:
   * where the community is headed ​   * where the community is headed ​
   * help identify potential bottlenecks   * help identify potential bottlenecks
- 
-[[@jgbarah]] has a tremendous post on how to derive the most meaningful metrics. 
-Please refer to it below. 
-[[http://​www.communityleadershipforum.com/​t/​7b-analytics-metrics-for-community-based-software-development/​268]] 
  
 In this post we will identify the challenges, pitfalls to avoid and guidelines to arrive at the metrics for a community. ​ In this post we will identify the challenges, pitfalls to avoid and guidelines to arrive at the metrics for a community. ​
  
-==== Recommended Approaches ====+==== Metrics & Community: ​Recommended Approaches ====
  
 == Preliminary Steps == == Preliminary Steps ==
  
-  - Even before ​we start collecting ​metrics ​we should consider the target audience for the metrics. Reports targeted at the engineering department will not fit the needs of marketing department. +  - Even before ​beginning to collect ​metrics ​one should consider the target audience for the metrics. ​ 
-  - We need to bear in mind that metrics ​are numbers that can be manipulated by gaming the system. +    * Reports targeted at the engineering department will not fit the needs of marketing department. 
-  - Remember the rule of participation:​ 90% of people are lurkers, 9% contributors,​ 1% creators. +  - Metrics ​are numbers that can be manipulated by gaming the system. 
-  ​- ​[[http://​en.wikipedia.org/​wiki/​1%25_rule_%28Internet_culture%29]]+  - The rule of participation:​ 90% of people are lurkers, 9% contributors,​ 1% creators. 
 +    ​* ​[[http://​en.wikipedia.org/​wiki/​1%25_rule_%28Internet_culture%29]]
  
 == Sources of Metrics == == Sources of Metrics ==
Line 44: Line 41:
   * User feedback   * User feedback
     * Detecting lack of features, by tracking user actions and receiving user feedback.     * Detecting lack of features, by tracking user actions and receiving user feedback.
-  * Deplying ​programs on the community server.  +  * Deploying ​programs on the community server.  
-    * Feedback ​to customer support portal.+    * Send the user feedback ​to customer support portal.
  
 == Condensed list of Metrics == == Condensed list of Metrics ==
  
   * Q & A from forum or mailing list.   * Q & A from forum or mailing list.
-  * How many returns versus new logins.+  * Community Portal - How many returns versus new logins.
   * Average time between when a bug is submitted and it is fixed.   * Average time between when a bug is submitted and it is fixed.
   * Average time for a contribution to be entered into the code base.    * Average time for a contribution to be entered into the code base. 
Line 63: Line 60:
     * Regressions     * Regressions
     * Other code quality issues     * Other code quality issues
 +  * How often code extracts are making it into other projects (Example : GitHub)
 +  * Core project versus side project.
 +    * Who is helping with the existing code?
 +    * Who is contributing a new feature?
 +    * Why weight the different groups?
 +    * How to assign weights for contributions?​
 +  * Consider two very different cases of contributions
 +    * individual, independent volunteer contributions
 +    * developers hired by a company ​
  
-=== Points of Interest to Community Managers === +**What actions ​should ​the community ​manager take based on the above?*
-Tracking the actions ​online, and offline, and matching both. +//By-laws of the project should be careful ​about cases where companies are trying to influence ​the project versus cases where companies adopt a passive stance.//
-    * How is the community's online visibility and presence influencing its accessibility. +
-    ​How are Meetup groups helping your community. +
-    * How many people joined the community online because of an in-person event. +
-    * How many people got to know a contact point from the community via online activity (StackOverflow,​ reddit etc.). +
-This gives the community managers a sense of the diversity of their community and says a lot about the appeal of the community.+
  
  
-Meetup groups: for how many people the first point of contact was an in-person event, and now they are joining the online group? +==== Pitfalls To Avoid ====
-Or just knowing the first point of contact, if it was asking and maybe answering questions on StackOverflow+
  
-Bugs?  Regressions?​ Other code quality issues?+== Why Collect Metrics == 
 +  * Protect the project from management. 
 +  * Know what is going on helps them make smarter decisions. 
 +  * Management or PR department need numbers to prove estimated outcomes. 
 +    * The business perspective of community. 
 +  * Focusing on which companies are contributing the most, that can cause land-grabbing
  
-How often code extracts are making it into other projects ​(would love to get this). GitHub ​can do this+For open source, numbers ​(and transparency in numbersshould be important. 
 +Metrics ​can inadvertently polarize communities with questions like, 
 +  * Developers are important?​ 
 +  * Communities are important?​ 
 +So, use them judiciously and transparently.
  
-Core project versus side projectSplit again - who is helping ​with the existing code, or who is contributing a new featureWhy weight ​the different groups? ​ +==== Points of Interest ==== 
 +  * Tracking the actions online, and offline, and matching both. 
 +    * How is the community'​s online visibility and presence influencing its accessibility. 
 +    * How are Meetup groups ​helping ​your community. 
 +    * How many people joined ​the community online because of an in-person event. 
 +    * How many people got to know a contact point from the community via online activity (StackOverflow,​ reddit etc.). 
 +  * What causes spikes/dips in contributions?​ 
 +    * why does it take long time for people to contribute?  
 +    * Is it ramp up time? 
 +    * Lack of Training? ​
  
-What actions to take based on the above.+This gives the community managers a sensing report. 
 +It also helps understand which online channels are benefitting the community and what can be improved.
  
 +==== Tools ====
  
-    Community Managers protect the project from managementTheir asking for data is usually just to prove their stance +  * [[http://​google.com/​analytics|Google Analytics]] 
-    There are two very different casesindividual, independent volunteer developers commit the code, or developers hired by a company are the ones making contributions,​ This can be a large difference with respect to how the project is performing.+  * [[http://analytics.twitter.com|Twitter Analytics]] 
 +  * [[http://​www.openhub.net|OpenHub]] 
 +  * [[http://​metricsgrimoire.github.io|Metrics Grimoire toolkit]] 
 +  * [[http://​biterg.io|The Cauldron]](//​beta//​) 
 +  * ... Feel free to add more
  
-"why collect information of this kind?" 
- 
- 
-    Know what is going on to make smarter decisions. Usually community managers are interested in this 
-    I have an outcome in mind, I can prove it using metrics. Usually management or the PR department want this 
- 
-For some open source projects (ie Apache), in some cases they try not to consider too much the company hiring each developer, consider they speak on their own behalf. There is a balance between companies riding the project and companies not being important for it. By-laws of the project should be careful here. 
- 
-Focusing on which companies are contributing the most, that can cause land-grabbing 
- 
-Case found in the Drupal Community. What was causing some spikes in contributions? ​ 
- 
-    The increase in participation happens about three months after their community event. Now that you know that, why does it take so long? Ramp up time?  Training? 
-    The metric had to be looked at a few months after the sprint, then you need continuous tracking. 
- 
-Some final thoughts: 
- 
-    For open source, numbers (and transparency in numbers) should be important. 
- 
-    Developers are important; or communities are important? 
- 
- 
- 
- 
- 
-=== Approach 2 === 
- 
-One approach. Be detailed in how to apply this method of approaching the topic - where possible, provide practical steps. 
- 
-==== Pitfalls To Avoid ==== 
- 
-Provide some guidance of common pitfalls and problems to avoid. 
- 
-==== Tips and Tricks ==== 
- 
-Provide some guidance of tips and tricks to bear in mind when approaching this topic. 
  
 ==== Further Reading ==== ==== Further Reading ====
  
-Provide ​bullet list of addition pages to look at our additional resources. +  * [[http://​twitter.com/​jgbarah|@jgbarah]] has [[http://​www.communityleadershipforum.com/​t/​7b-analytics-metrics-for-community-based-software-development/​268|tremendous post]] on how to derive the most meaningful metrics
- +  //​[[http://​jgbarah.gitbooks.io/​evaluating-foss-projects/​|Evaluating Free / Open Source Software Projects]]//​ gitbook from Jesús MGonzalez-Barahona covers several areas related with community metrics
- * Something ​to look at+
- ​* ​Something to look at.+