Detailed Conference Program

June 2, 09:00-09:15 — Preface

• Welcome from the Chairs
  Michele Lanza, Massimiliano Di Penta, and Tao Xie
  (University of Lugano, Switzerland; University of Sannio, Italy; North Carolina State University, USA)

June 2, 11:00-12:30 — Session I — Software Repositories

Session chair: Tom Zimmermann

• Towards Improving Bug Tracking Systems with Game Mechanisms
  Rafael Lotufo, Leonardo Passos, and Krzysztof Czarnecki
  (University of Waterloo, Canada)
  Abstract
  Low bug report quality and human conflicts pose challenges to keep bug tracking systems productive. This work proposes to address these issues by applying game mechanisms to bug tracking systems. We investigate the use of game mechanisms in Stack Overflow, an online community organized to resolve computer programming related problems, for which the improvements we seek for bug tracking systems also turn out to be relevant. The results of our Stack Overflow investigation show that its game mechanisms could be used to address these issues by motivating contributors to increase contribution frequency and quality, by filtering useful contributions, and by creating an agile and dependable moderation system. We proceed by mapping these mechanisms to open-source bug tracking systems, and find that most benefits are applicable. Additionally, our results motivate tailoring a reward and reputation system and summarizing bug reports as future directions for increasing the benefits of game mechanisms in bug tracking systems.
• GHTorrent: Github’s Data from a Firehose
  Georgios Gousios and Diomidis Spinellis
  (Athens University of Economics and Business, Greece)
  Abstract
  A common requirement of many empirical software engineering studies is the acquisition and curation of data from software repositories. During the last few years, GitHub has emerged as a popular project hosting, mirroring and collaboration platform. GitHub provides an extensive REST API, which enables researchers to retrieve both the commits to the projects’ repositories and events generated through user actions on project resources. GHTorrent aims to create a scalable off line mirror of GitHub’s event streams and persistent data, and offer it to the research community as a service. In this paper, we present the project’s design and initial implementation and demonstrate how the provided datasets can be queried and processed.
• Explaining Software Defects Using Topic Models
  Tse-Hsun Chen, Stephen W. Thomas, Meiyappan Nagappan, and Ahmed E. Hassan
  (Queen’s University, Canada)
  Abstract
  Researchers have proposed various metrics based on measurable aspects of the source code entities (e.g., methods, classes, files, or modules) and the social structure of a software project in an effort to explain the relationships between software development and software defects.However, these metrics largely ignore the actual functionality, i.e., the conceptual concerns, of a software system, which are the main technical concepts that reflect the business logic or domain of the system. For instance, while lines of code may be a good general measure for defects, a large entity responsible for simple I/O tasks is likely to have fewer defects than a small entity responsible for complicated compiler implementation details. In this paper, we study the effect of conceptual concerns on code quality. We use a statistical topic modeling technique to approximate software concerns as topics; we then propose various metrics on these topics to help explain the defect-proneness (i.e., quality) of the entities. Paramount to our proposed metrics is that they take into account the defect history of each topic. Case studies on multiple versions of Mozilla Firefox, Eclipse, and Mylyn show that (i) some topics are much more defect-prone than others, (ii) defect-prone topics tend to remain so over time, and (iii) defect-prone topics provide additional explanatory power for code quality over existing structural and historical metrics.
• A Linked Data Platform for Mining Software Repositories (short paper)
  Iman Keivanloo, Christopher Forbes, Aseel Hmood, Mostafa Erfani, Christopher Neal, George Peristerakis, and Juergen Rilling
  (Concordia University, Canada)
  Abstract
  The mining of software repositories involves the extraction of both basic and value-added information from existing software repositories. The repositories will be mined to extract facts by different stakeholders (e.g. researchers, managers) and for various purposes. To avoid unnecessary pre-processing and analysis steps, sharing and integration of both basic and value-added facts are needed. In this research, we introduce SeCold, an open and collaborative platform for sharing software datasets. SeCold provides the first online software ecosystem Linked Data platform that supports data extraction and on-the-fly inter-dataset integration from major version control, issue tracking, and quality evaluation systems. In its first release, the dataset contains about two billion facts, such as source code statements, software licenses, and code clones from 18 000 software projects. In its second release the SeCold project will contain additional facts mined from issue trackers and versioning systems. Our approach is based on the same fundamental principle as Wikipedia: researchers and tool developers share analysis results obtained from their tools by publishing them as part of the SeCold portal and therefore make them an integrated part of the global knowledge domain. The SeCold project is an official member of the Linked Data dataset cloud and is currently the eighth largest online dataset available on the Web.
• How Distributed Version Control Systems Impact Open Source Software Projects (short paper)
  Christian Rodríguez-Bustos and Jairo Aponte
  (Universidad Nacional de Colombia, Colombia)
  Abstract
  Centralized Version Control Systems have been used by many open source projects for a long time. However, in recent years several widely-known projects have migrated their repositories to Distributed Version Control Systems, such as Mercurial, Bazaar, and Git. Such systems have technical features that allow contributors to work in new ways, as various different workflows are possible. We plan to study this migration process to assess how developers’ organization and their contributions are affected. As a first step, we present an analysis of the Mozilla repositories, which migrated from CVS to Mercurial in 2007. This analysis reveals both expected and unexpected aspects of the contributors’ activities.



June 2, 14:00-15:30 — Session II — Bug Fixing and Prediction

Session chair: Harald C. Gall



• An Empirical Study of Supplementary Bug Fixes
  Jihun Park, Miryung Kim, Baishakhi Ray, and Doo-Hwan Bae
  (KAIST, South Korea; University of Texas at Austin, USA)
  Abstract
  A recent study finds that errors of omission are harder for programmers to detect than errors of commission. While several change recommendation systems already exist to prevent or reduce omission errors during software development, there have been very few studies on why errors of omission occur in practice and how such errors could be prevented. In order to understand the characteristics of omission errors, this paper investigates a group of bugs that were fixed more than once in open source projects - those bugs whose initial patches were later considered incomplete and to which programmers applied supplementary patches. Our study on Eclipse JDT core, Eclipse SWT, and Mozilla shows that a significant portion of resolved bugs (22% to 33%) involves more than one fix attempt. Our manual inspection shows that the causes of omission errors are diverse, including missed porting changes, incorrect handling of conditional statements, or incomplete refactorings, etc. While many consider that missed updates to code clones often lead to omission errors, only a very small portion of supplementary patches (15% in JDT, 17% in SWT, and 8% in Mozilla) have a content similar to their initial patches. This implies that supplementary change locations cannot be predicted by code clone analysis alone. Furthermore, 14% to 15% of files in supplementary patches are beyond the scope of immediate neighbors of their initial patch locations - they did not overlap with the initial patch locations nor had direct structural dependences on them (e.g. calls, accesses, subtyping relations, etc.). These results call for new types of omission error prevention approaches that complement existing change recommendation systems.
• Incorporating Version Histories in Information Retrieval Based Bug Localization
  Bunyamin Sisman and Avinash Kak
  (Purdue University, USA)
  Abstract
  Fast and accurate localization of software defects continues to be a difficult problem since defects can emanate from a large variety of sources and can often be intricate in nature. In this paper, we show how version histories of a software project can be used to estimate a prior probability distribution for defect proneness associated with the files in a given version of the project. Subsequently, these priors are used in an IR (Information Retrieval) framework to determine the posterior probability of a file being the cause of a bug. We first present two models to estimate the priors, one from the defect histories and the other from the modification histories, with both types of histories as stored in the versioning tools. Referring to these as the base models, we then extend them by incorporating a temporal decay into the estimation of the priors. We show that by just including the base models, the mean average precision (MAP) for bug localization improves by as much as 30%. And when we also factor in the time decay in the estimates of the priors, the improvements in MAP can be as large as 80%.
• Think Locally, Act Globally: Improving Defect and Effort Prediction Models
  Nicolas Bettenburg, Meiyappan Nagappan, and Ahmed E. Hassan
  (Queen’s University, Canada)
  Abstract
  Much research energy in software engineering is focused on the creation of effort and defect prediction models. Such models are important means for practitioners to judge their current project situation, optimize the allocation of their resources, and make informed future decisions. However, software engineering data contains a large amount of variability. Recent research demonstrates that such variability leads to poor fits of machine learning models to the underlying data, and suggests splitting datasets into more fine-grained subsets with similar properties. In this paper, we present a comparison of three different approaches for creating statistical regression models to model and predict software defects and development effort. Global models are trained on the whole dataset. In contrast, local models are trained on subsets of the dataset. Last, we build a global model that takes into account local characteristics of the data. We evaluate the performance of these three approaches in a case study on two defect and two effort datasets. We find that for both types of data, local models show a significantly increased fit to the data compared to global models. The substantial improvements in both relative and absolute prediction errors demonstrate that this increased goodness of fit is valuable in practice. Finally, our experiments suggest that trends obtained from global models are too general for practical recommendations. At the same time, local models provide a multitude of trends which are only valid for specific subsets of the data. Instead, we advocate the use of trends obtained from global models that take into account local characteristics, as they combine the best of both worlds.
• Characterizing Verification of Bug Fixes in Two Open Source IDEs (short paper)
  Rodrigo Souza and Christina Chavez
  (Federal University of Bahia, Brazil)
  Abstract
  Data from bug repositories have been used to enable inquiries about software product and process quality. Unfortunately, such repositories often contain inaccurate, inconsistent, or missing data, which can originate misleading results. In this paper, we investigate how well data from bug repositories support the discovery of details about the software verification process in two open source projects, Eclipse and NetBeans. We have been able do identify quality assurance teams in NetBeans and to detect a well-defined verification phase in Eclipse. A major challenge, however, was to identify the verification techniques used in the projects. Moreover, we found cases in which a large batch of bug fixes is simultaneously reported to be verified, although no software verification was actually done. Such mass verifications, if not acknowledged, threatens analyses that rely on information about software verification reported on bug repositories. Therefore, we recommend that the exploratory analyses presented in this paper precede inferences based on reported verifications.
• Are Faults Localizable? (short paper)
  Lucia, Ferdian Thung, David Lo, and Lingxiao Jiang
  (Singapore Management University, Singapore)
  Abstract
  Many fault localization techniques have been proposed to facilitate debugging activities. Most of them attempt to pinpoint the location of faults (i.e., localize faults) based on a set of failing and correct executions and expect debuggers to investigate a certain number of located program elements to find faults. These techniques thus assume that faults are localizable, i.e., only one or a few lines of code that are close to one another are responsible for each fault. However, in reality, are faults localizable? In this work, we investigate hundreds of real faults in several software systems, and find that many faults may not be localizable to a few lines of code and these include faults with high severity level.



June 2, 16:00-17:15 — Session III — New MSR Trends

Session chair: Ahmed E. Hassan



• Green Mining: A Methodology of Relating Software Change to Power Consumption
  Abram Hindle
  (University of Alberta, Canada)
  Abstract
  Power consumption is becoming more and more important with the increased popularity of smart-phones, tablets and laptops. The threat of reducing a customer’s battery-life now hangs over the software developer who asks, “will this next change be the one that causes my software to drain a customer’s battery?” One solution is to detect power consumption regressions by measuring the power usage of tests, but this is time-consuming and often noisy. An alternative is to rely on software metrics that allow us to estimate the impact that a change might have on power consumption thus relieving the developer from expensive testing. This paper presents a general methodology for investigating the impact of software change on power consumption, we relate power consumption to software changes, and then investigate the impact of static OO software metrics on power consumption. We demonstrated that software change can effect power consumption using the Firefox web-browser and the Azeurus/Vuze BitTorrent client. We found evidence of a potential relationship between some software metrics and power consumption. In conclusion, we explored the effect of software change on power consumption on two projects; and we provide an initial investigation on the impact of software metrics on power consumption.
• Analysis of Customer Satisfaction Survey Data
  Pete Rotella and Sunita Chulani
  (Cisco Systems, USA)
  Abstract
  Cisco Systems, Inc., conducts a customer satisfaction survey (CSAT) each year to gauge customer sentiment regarding Cisco products, technical support, partner- and Cisco-provided technical services, order fulfillment, and a number of other aspects of the company’s business. The results of the analysis of this data are used for several purposes, including ascertaining the viability of new products, determining if customer support objectives are being met, setting engineering in-process and customer experience yearly metrics goals, and assessing, indirectly, the success of engineering initiatives. We also use the CSAT results as a multiplying factor in the equation used to calculate yearly bonuses for all employees. Since many corporate investment and prioritization decisions are largely based on this specific form of customer feedback, it is essential that the analysis of the survey data is conducted with mathematical rigor, to give management an accurate picture of the current state of customer sentiment and associated trends, including accurate statistical confidence intervals around these. Analyzing this data, which includes 110,000 yearly sets of survey responses that address over 100 product and services categories, is in many respects complicated. For example, skip logic is an integral part of the survey mechanics, and forming aggregate views of customer sentiment is statistically challenging in this data environment. In this paper, we describe several of the various analysis approaches currently used, pointing out some situations where a high level of precision is not easily achieved, and some situations in which it is possible to easily end up with erroneous results. The analysis and statistical territory covered in this paper is in parts well-known and straightforward, but other parts, which we address, are susceptible to large inaccuracies and errors. We address several of these difficulties and develop reasonable solutions for two known issues, high missing value levels and high colinearity of independent variables.
• Mining Usage Data and Development Artifacts
  Olga Baysal, Reid Holmes, and Michael W. Godfrey
  (University of Waterloo, Canada)
  Abstract
  Software repository mining techniques generally focus on analyzing, unifying, and querying different kinds of development artifacts, such as source code, version control meta-data, defect tracking data, and electronic communication. In this work, we demonstrate how adding real-world usage data enables addressing broader questions of how software systems are actually used in practice, and by inference how development characteristics ultimately affect deployment, adoption, and usage. In particular, we explore how usage data that has been extracted from web server logs can be unified with product release history to study questions that concern both users’ detailed dynamic behaviour as well as broad adoption trends across different deployment environments. To validate our approach, we performed a study of two open source web browsers: Firefox and Chrome. We found that while Chrome is being adopted at a consistent rate across platforms, Linux users have an order of magnitude higher rate of Firefox adoption. Also, Firefox adoption has been concentrated mainly in North America, while Chrome users appear to be more evenly distributed across the globe. Finally, we detected no evidence in age-specific differences in navigation behaviour among Chrome and Firefox users; however, we hypothesize that younger users are more likely to have more up-to-date versions than more mature users.
• App Store Mining and Analysis: MSR for App Stores (short paper)
  Mark Harman, Yue Jia, and Yuanyuan Zhang
  (University College London, UK)
  Abstract
  This paper introduces app store mining and analysis as a form of software repository mining. Unlike other software repositories traditionally used in MSR work, app stores usually do not provide source code. However, they do provide a wealth of other information in the form of pricing and customer reviews. Therefore, we use data mining to extract feature information, which we then combine with more readily available information to analyse apps’ technical, customer and business aspects. We applied our approach to the 32,108 non-zero priced apps available in the Blackberry app store in September 2011. Our results show that there is a strong correlation between customer rating and the rank of app downloads, though perhaps surprisingly, there is no correlation between price and downloads, nor between price and rating. More importantly, we show that these correlation findings carry over to (and are even occasionally enhanced within) the space of data mined app features, providing evidence that our ‘App store MSR’ approach can be valuable to app developers.



June 2, 17:15-18:00 — Mining Challenge

Session chair: Emad Shihab



• Bug Introducing Changes: A Case Study with Android
  Muhammad Asaduzzaman, Michael Bullock, Chanchal K. Roy, and Kevin A. Schneider
  (University of Saskatchewan, Canada)
  Abstract
  Changes, a rather inevitable part of software development can cause maintenance implications if they introduce bugs into the system. By isolating and characterizing these bug introducing changes it is possible to uncover potential risky source code entities or issues that produce bugs. In this paper, we mine the bug introducing changes in the Android platform by mapping bug reports to the changes that introduced the bugs. We then use the change information to look for both potential problematic parts and dynamics in development that can cause maintenance implications. We believe that the results of our study can help better manage Android software development.
• Trendy Bugs: Topic Trends in the Android Bug Reports
  Lee Martie, Vijay Krishna Palepu, Hitesh Sajnani, and Cristina Lopes
  (UC Irvine, USA)
  Abstract
  Studying vast volumes of bug and issue discussions can give an understanding of what the community has been most concerned about, however the magnitude of documents can overload the analyst. We present an approach to analyze the development of the Android open source project by observing trends in the bug discussions in the Android open source project public issue tracker. This informs us of the features or parts of the project that are more problematic at any given point of time. In turn, this can be used to aid resource allocation (such as time and man power) to parts or features. We support these ideas by presenting the results of issue topic distributions over time using statistical analysis of the bug descriptions and comments for the Android open source project. Furthermore, we show relationships between those time distributions and major development releases of the Android OS.
• Do the Stars Align? Multidimensional Analysis of Android’s Layered Architecture
  Victor Guana, Fabio Rocha, Abram Hindle, and Eleni Stroulia
  (University of Alberta, Canada)
  Abstract
  In this paper we mine the Android bug tracker repository and study the characteristics of the architectural layers of the Android system. We have identified the locality of the Android bugs in the architectural layers of the its infrastructure, and analysed the bug lifetime patterns in each one of them. Additionally, we mined the bug tracker reporters and classified them according to its social centrality in the Android bug tracker community. We report three interesting findings, firstly while some architectural layers have a diverse interaction of people, attracting not only non-central reporters but highly important ones, other layers are mostly captivating for peripheral actors. Second, we exposed that even the bug lifetime is similar across the architectural layers, some of them have higher bug density and differential percentages of unsolved bugs. Finally, comparing the popularity distribution between layers, we have identified one particular layer that is more important to developers and users alike.
• The Build Dependency Perspective of Android’s Concrete Architecture
  Wei Hu, Dan Han, Abram Hindle, and Kenny Wong
  (University of Alberta, Canada)
  Abstract
  Android is an operating system designed specifically for mobile devices. It has a layered architecture. In this paper, we extract Android&rqsuo;s concrete layered architecture by analyzing the build dependency relation between Android sub-projects and use it to validate the proposed conceptual architecture. Our experiment shows that Android&rqsuo;s concrete architecture conforms to the conceptual architecture. Apart from that, we also show the extracted architecture can help developers and users better understand the Android system and further demonstrate its potential benefits in studying the impact of changes.
• MINCE: MINing ChangE History of Android Project
  Vibha Sinha, Senthil Mani, and Monika Gupta
  (IBM Research, India)
  Abstract
  An analysis of commit history of Android reveals that Android has a code base of 550K files, where on an average each file has been modified 8.7 times. 41% of files have been modified at-least once. In terms of contributors, it has an overall contributor community of 1563, with 58.5% of them having made >5 commits. Moreover, the contributor community shows high churn levels, with only 13 of contributors continuing from 2005 to 2011. In terms of industry participation, Google & Android account for 22% of developers. Intel and RedHat account for 2% of contributors each and IBM, Oracle, TI, SGI account for another 1% each. Android code can be classified into 5 sub-projects: kernel, platform, device, tools and toolchain. In this paper, we profile each of these sub-projects in terms of change volumes, contributor and industry participation. We further picked specific framework topics such as UI, security, whose understanding is required from perspective of developing apps over Android, and present some insights on community participation around the same.
• Mining for Localization in Android
  Laura Arjona Reina and Gregorio Robles
  (Universidad Rey Juan Carlos, Spain)
  Abstract
  Localization, and in particular translation, is a key aspect of modern end-user software applications. Open source systems have traditionally taken advantage of distributed and volunteer collaboration to carry localization tasks. In this paper, we will analyze the Android source code repository to know how localization and translation is managed: who participates in this kind of tasks, if the translation workflows, participants and processes follow the same patterns as the rest of the development, and if the Android project takes benefit from external contributions. Our results show that Android should ease the localization tasks to benefit from external contributions. Steps towards obtaining a specialized team as found in many other free software projects are also encouraged.



June 3, 11:00-12:30 — Session IV — Software Analysis

Session chair: Prem Devambu



• Why Do Software Packages Conflict?
  Cyrille Valentin Artho, Kuniyasu Suzaki, Roberto Di Cosmo, Ralf Treinen, and Stefano Zacchiroli
  (AIST, Japan; Paris Diderot University, France; INRIA, France)
  Abstract
  Determining whether two or more packages cannot be installed together is an important issue in the quality assurance process of package-based distributions. Unfortunately, the sheer number of different configurations to test makes this task particularly challenging, and hundreds of such incompatibilities go undetected by the normal testing and distribution process until they are later reported by a user as bugs that we call “conflict defects”. We performed an extensive case study of conflict defects extracted from the bug tracking systems of Debian and Red Hat. According to our results, conflict defects can be grouped into five main categories. We show that with more detailed package meta-data, about 30~\% of all conflict defects could be prevented relatively easily, while another 30~\% could be found by targeted testing of packages that share common resources or characteristics. These results allow us to make precise suggestions on how to prevent and detect conflict defects in the future.
• Discovering Complete API Rules with Mutation Testing
  Anh Cuong Nguyen and Siau-Cheng Khoo
  (National University of Singapore, Singapore)
  Abstract
  Specifications are important for many activities during software construction and maintenance process such as testing, verification, debugging and repairing. Despite their importance, specifications are often missing, informal or incomplete because they are difficult to write manually. Many techniques have been proposed to automatically mine specifications describing method call sequence from execution traces or source code using frequent pattern mining. Unfortunately, a sizeable number of such “interesting” specifications discovered by frequent pattern mining may not capture the correct use patterns of method calls. Consequently, when used in software testing or verification, these mined specifications lead to many false positive defects, which in turn consume much effort for manual investigation. We present a novel framework for automatically discovering legitimate specifications from execution traces using a mutation testing based approach. Such an approach gives a semantics bearing to the legitimacy of the discovered specifications. We introduce the notion of maximal precision and completeness as the desired forms of discovered specifications, and describe in detail suppression techniques that aid efficient discovery. Preliminary evaluation of this approach on several open source software projects shows that specifications discovered through our approach, compared with those discovered through frequent pattern mining, are much more precise and complete. When used in finding bugs, our specifications also locate defects with significantly fewer false positives and more true positives.
• Inferring Semantically Related Words from Software Context
  Jinqiu Yang and Lin Tan
  (University of Waterloo, Canada)
  Abstract
  Code search is an integral part of software development and program comprehension. The difficulty of code search lies in the inability to guess the exact words used in the code. Therefore, it is crucial for keyword-based code search to expand queries with semantically related words, e.g., synonyms and abbreviations, to increase the search effectiveness. However, it is limited to rely on resources such as English dictionaries and WordNet to obtain semantically related words in software, because many words that are semantically related in software are not semantically related in English. This paper proposes a simple and general technique to automatically infer semantically related words in software by leveraging the context of words in comments and code. We achieve a reasonable accuracy in seven large and popular code bases written in C and Java. Our further evaluation against the state of art shows that our technique can achieve a higher precision and recall.
• The Evolution of Data Races (short paper)
  Caitlin Sadowski, Jaeheon Yi, and Sunghun Kim
  (UC Santa Cruz, USA; Hong Kong University of Science and Technology, China)
  Abstract
  Concurrency bugs are notoriously difficult to find and fix. Several prior empirical studies have identified the prevalence and challenges of concurrency bugs in open source projects, and several existing tools can be used to identify concurrency errors such as data races. However, little is known about how concurrency bugs evolve over time. In this paper, we examine the evolution of data races by analyzing samples of the committed code in two open source projects over a multi-year period. Specifically, we identify how the data races in these programs change over time.
• Error Mining: Bug Detection through Comparison with Large Code Databases (short paper)
  Alexander Breckel
  (University of Ulm, Germany)
  Abstract
  Bugs are hard to find. Static analysis tools are capable of systematically detecting predefined sets of errors, but extending them to find new error types requires a deep understanding of the underlying programming language. Manual reviews on the other hand, while being able to reveal more individual errors, require much more time. We present a new approach to automatically detect bugs through comparison with a large code database. The source file is analyzed for similar but slightly different code fragments in the database. Frequent occurrences of common differences indicate a potential bug that can be fixed by applying the modification back to the original source file. In this paper, we give an overview of the resulting algorithm and some important implementation details. We further evaluate the circumstances under which good detection rates can be achieved. The results demonstrate that consistently high detection rates of up to 50% are possible for certain error types across different programming languages.



June 3, 14:00-15:30 — Session V — Quality & Performance

Session chair: Abram Hindle



• Do Faster Releases Improve Software Quality? An Empirical Case Study of Mozilla Firefox
  Foutse Khomh, Tejinder Dhaliwal, Ying Zou, and Bram Adams
  (Queen’s University, Canada; École Polytechnique de Montréal, Canada)
  Abstract
  Nowadays, many software companies are shifting from the traditional 18-month release cycle to shorter release cycles. For example, Google Chrome and Mozilla Firefox release new versions every 6 weeks. These shorter release cycles reduce the users’ waiting time for a new release and offer better marketing opportunities to companies, but it is unclear if the quality of the software product improves as well, since shorter release cycles result in shorter testing periods. In this paper, we empirically study the development process of Mozilla Firefox in 2010 and 2011, a period during which the project transitioned to a shorter release cycle. We compare crash rates, median uptime, and the proportion of post-release bugs of the versions that had a shorter release cycle with those having a traditional release cycle, to assess the relation between release cycle length and the software quality observed by the end user. We found that (1) with shorter release cycles, users do not experience significantly more post-release bugs and (2) bugs are fixed faster, yet (3) users experience these bugs earlier during software execution (the program crashes earlier).
• MIC Check: A Correlation Tactic for ESE Data
  Daryl Posnett, Prem Devanbu, and Vladimir Filkov
  (UC Davis, USA)
  Abstract
  Empirical software engineering researchers are concerned with understanding the relationships between outcomes of interest, e.g. defects, and process and product measures. The use of correlations to uncover strong relationships is a natural precursor to multivariate modeling. Unfortunately, correlation coefficients can be difficult and/or misleading to interpret. For example, a strong correlation occurs between variables that stand in a polynomial relationship; this may lead one mistakenly, and eventually misleadingly, to model a polynomially related variable in a linear regression. Likewise, a non-monotonic functional, or even non-functional relationship might be entirely missed by a correlation coefficient. Outliers can influence standard correlation measures, tied values can unduly influence even robust non-parametric rank correlation,measures, and smaller sample sizes can cause instability in correlation measures. A new bivariate measure of association, Maximal Information Coefficient (MIC), promises to simultaneously discover if two variables have: a) any association, b) a functional relationship, and c) a non-linear relationship. The MIC is a very useful complement to standard rank correlation measures. It separately characterizes the existence of a relationship and its precise nature; thus, it enables more informed choices in modeling non-functional and non-linear relationships, and a more nuanced indicator of potential problems with the values reported by standard and rank correlation measures. We illustrate the use of MIC using a variety of software engineering metrics. We study and explain the distributional properties of MIC and related measures in software engineering data, and illustrate the value of these measures for the empirical software engineering researcher.
• A Qualitative Study on Performance Bugs
  Shahed Zaman, Bram Adams, and Ahmed E. Hassan
  (Queen’s University, Canada; École Polytechnique de Montréal, Canada)
  Abstract
  Software performance is one of the important qualities that makes software stand out in a competitive market. However, in earlier work we found that performance bugs take more time to fix, need to be fixed by more experienced developers and require changes to more code than non-performance bugs. In order to be able to improve the resolution of performance bugs, a better understanding is needed of the current practice and shortcomings of reporting, reproducing, tracking and fixing performance bugs. This paper qualitatively studies a random sample of 400 performance and non-performance bug reports of Mozilla Firefox and Google Chrome across four dimensions (Impact, Context, Fix and Fix validation). We found that developers and users face problems in reproducing performance bugs and have to spend more time discussing performance bugs than other kinds of bugs. Sometimes performance regressions are tolerated as a trade-off to improve something else.
• Issue Handling Performance in Proprietary Software Projects (short paper)
  Aigerim Issabayeva, Ariadi Nugroho, and Joost Visser
  (Tilburg University, Netherlands; Software Improvement Group, Netherlands)
  Abstract
  Software maintenance tasks are mainly related to fixing defects and implementing new features. Higher efficiency in performing such tasks is therefore going to reduce the costs of maintenance. A previous study involving open source systems has shown that higher software maintainability corresponds to faster speed in fixing defects (Luijten, 2010). In this paper we replicate the previous study by mining bug report data of three proprietary software projects. In one of the projects, a correlation between higher software maintainability and faster defect resolution is confirmed. The quality of issue handling process (e.g., issue registration accuracy and completeness, scope and complexity of issue workflow) should be considered in further research as it might explain the circumstances under which the correlation can be observed.
• Co-evolution of Logical Couplings and Commits for Defect Estimation (short paper)
  Maximilian Steff and Barbara Russo
  (Free University of Bolzano, Italy)
  Abstract
  Logical couplings between files in the commit history of a software repository are instances of files being changed together. The evolution of couplings over commits’ history has been used for the localization and prediction of software defects in software reliability. Couplings have been represented in class graphs and change histories on the class-level have been used to identify defective modules. Our new approach inverts this perspective and constructs graphs of ordered commits coupled by common changed classes. These graphs, thus, represent the co-evolution of commits, structured by the change patterns among classes. We believe that co-evolutionary graphs are a promising new instrument for detecting defective software structures. As a first result, we have been able to correlate the history of logical couplings to the history of defects for every commit in the graph and to identify sub-structures of bug-fixing commits over sub-structures of normal commits.



June 3, 16:00-17:30 — Session VI — Evolution and Process

Session chair: Michael W. Godfrey



• Can We Predict Types of Code Changes? An Empirical Analysis
  Emanuel Giger, Martin Pinzger, and Harald C. Gall
  (University of Zurich, Switzerland; TU Delft, Netherlands)
  Abstract
  There exist many approaches that help in pointing developers to the change-prone parts of a software system. Although beneficial, they mostly fall short in providing details of these changes. Fine-grained source code changes (SCC) capture such detailed code changes and their semantics on the statement level. These SCC can be condition changes, interface modifications, inserts or deletions of methods and attributes, or other kinds of statement changes. In this paper, we explore prediction models for whether a source file will be affected by a certain type of SCC. These predictions are computed on the static source code dependency graph and use social network centrality measures and object-oriented metrics. For that, we use change data of the Eclipse platform and the Azureus 3 project. The results show that Neural Network models can predict categories of SCC types. Furthermore, our models can output a list of the potentially change-prone files ranked according to their change-proneness, overall and per change type category.
• An Empirical Investigation of Changes in Some Software Properties Over Time
  Yossi Gil, Maayan Goldstein, and Dany Moshkovich
  (Technion, Israel; IBM Research, Israel)
  Abstract
  Software metrics are easy to define, but not so easy to justify. It is hard to prove that a metric is valid, i.e., that measured numerical values imply anything on the vaguely defined, yet crucial software properties such as complexity and maintainability. This paper employs statistical analysis and tests to check some plausible assumptions on the behavior of software and metrics measured for this software in retrospective on its versions evolution history. Among those are the reliability assumption implicit in the application of any code metric, and the assumption that the magnitude of change, i.e., increase or decrease of its size, in a software artifact is correlated with changes to its version number. Putting a suite of 36 metrics to the trial, we confirm most of the assumptions on a large repository of software artifacts. Surprisingly, we show that a substantial portion of the reliability of some metrics can be observed even in random changes to architecture. Another surprising result is that Boolean-valued metrics tend to flip their values more often in minor software version increments than in major increments.
• Who? Where? What? Examining Distributed Development in Two Large Open Source Projects
  Christian Bird and Nachiappan Nagappan
  (Microsoft Research, USA; Microsoft, USA)
  Abstract
  To date, a large body of knowledge has been built up around understanding open source software development. However, there is limited research on examining levels of geo graphic and organizational distribution within open source software projects, despite many studies examining these same aspects in commercial contexts. We set out to fill this gap in OSS knowledge by manually collecting data for two large, mature, successful projects in an effort to assess how distributed they are, both geographically and organizationally. Both FIREFOX and ECLIPSE have been the subject of many studies and are ubiquitous in the areas of software development and internet usage respectively. Further, both receive substantial development contributions from many companies. As such, both are worthy of study in order to understand the development processes that they use, how distributed the projects are, and what, if any, relationship distribution has with quality. To this end, we identified the top contributors that made 95% of the changes over multiple major releases of FIREFOX and ECLIPSE and determined their geographic locations and organizational affiliations. We found that FIREFOX is very geographically distributed with over a third of its components receiving major contributions from developers on different continents, and that components that are highly distributed have no more defects than those that are not. In contrast, ECLIPSE is directed and developed largely by one company; with IBM making 96% of the total commits (49% coming from one lab in Ottawa, Canada). We further examined the distribution in each project’s constituent subsystems and report the relationship of pre- and post-release defects with geographic and organizational factors.
• What Does Software Engineering Community Microblog About? (short paper)
  Yuan Tian, Palakorn Achananuparp, Ibrahim Nelman Lubis, David Lo, and Ee-Peng Lim
  (Singapore Management University, Singapore)
  Abstract
  Microblogging is a new trend to communicate and to disseminate information. One microblog post could potentially reach millions of users. Millions of microblogs are generated on a daily basis on popular sites such as Twitter. The popularity of microblogging among programmers, software engineers, and software users has also led to their use of microblogs to communicate software engineering issues apart from using emails and other traditional communication channels. Understanding how millions of users use microblogs in software engineering related activities would shed light on ways we could leverage the fast evolving microblogging content to aid software development efforts. In this work, we perform a preliminary study on what the software engineering community microblogs about. We analyze the content of microblogs from Twitter and categorize the types of microblogs that are posted. We investigate the relative popularity of each category of microblogs. We also investigate what kinds of microblogs are diffused more widely in the Twitter network via the “retweet” feature. Our experiments show that microblogs commonly contain job openings, news, questions and answers, or links to download new tools and code. We find that microblogs concerning real-world events are more widely diffused in the Twitter network.
• Developing an H-Index for OSS Developers (short paper)
  Andrea Capiluppi, Alexander Serebrenik, and Ahmmad Youssef
  (University of East London, UK; Eindhoven University of Technology, Netherlands)
  Abstract
  The public data available in Open Source Software (OSS) repositories has been used for many practical reasons: detecting community structures; identifying key roles among developers; understanding software quality; predicting the arousal of bugs in large OSS systems, and so on; but also to formulate and validate new metrics and proof-of-concepts on general, non-OSS specific, software engineering aspects. One of the results that has not emerged yet from the analysis of OSS repositories is how to help the “career advancement” of developers: given the available data on products and processes used in OSS development, it should be possible to produce measurements to identify and describe a developer, that could be used externally as a measure of recognition and experience. This paper builds on top of the h-index, used in academic contexts, and which is used to determine the recognition of a researcher among her peers. By creating similar indices for OSS (or any) developers, this work could help defining a baseline for measuring and comparing the contributions of OSS developers in an objective, open and reproducible way.