Publications
2020
-
"Suggesting Comment Completions for Python Using Neural Language Models".
Adelina Ciurumelea,
Sebastian Proksch,
Harald C. Gall.
International Conference on Software Analysis, Evolution, and Reengineering.
2020.
...
Source-code comments are an important communication medium between developers to better understand and maintain software. Current research focuses on auto-generating comments by summarizing the code. However, good comments contain additional details, like important design decisions or required trade-offs, and only developers can decide on the proper comment content. Automated summarization techniques cannot include information that does not exist in the code, therefore fully-automated approaches while helpful, will be of limited use. In our work, we propose to empower developers through a semi-automated system instead. We investigate the feasibility of using neural language models trained on a large corpus of Python documentation strings to generate completion suggestions and obtain promising results. By focusing on confident predictions, we can obtain a top-3 accuracy of over 70%, although this comes at the cost of lower suggestion frequency. Our models can be improved by leveraging context information like the signature and the full body of the method. Additionally, we are able to return good accuracy completions even for new projects, suggesting the generalizability of our approach.
@inproceedings{CPG20-saner, title = {{Suggesting Comment Completions for Python Using Neural Language Models}}, author = {Adelina Ciurumelea AND Sebastian Proksch AND Harald C. Gall}, booktitle = {International Conference on Software Analysis, Evolution, and Reengineering}, year = 2020, }
2019
-
"Evo-Clocks: Software Evolution at a Glance".
Carol Alexandru,
Sebastian Proksch,
Pooyan Behnamghader,
Harald C. Gall.
Working Conference on Software Visualization.
2019.
...
Understanding the evolution of a project is crucial in reverse-engineering, auditing and otherwise understanding existing software. Visualizing how software evolves can be challenging, as it typically abstracts a multi-dimensional graph structure where individual components undergo frequent but localized changes. Existing approaches typically consider either only a small number of revisions or they focus on one particular aspect, such as the evolution of code metrics or architecture. Approaches using a static view with a time axis (such as line charts) are limited in their expressiveness regarding structure, and approaches visualizing structure quickly become cluttered with an increasing number of revisions and components. We propose a novel trade-off between displaying global structure over a large time period with reduced accuracy and visualizing fine-grained changes of individual components with absolute accuracy. We demonstrate how our approach displays changes by blending redundant visual features (such as scales or repeating data points) where they are not expressive. We show how using this approach to explore software evolution can reveal ephemeral information when familiarizing oneself with a new project. We provide a working implementation as an extension to our open-source framework for fine-grained evolution analysis, LISA.
@inproceedings{APBG19-vissoft, title = {{Evo-Clocks: Software Evolution at a Glance}}, author = {Carol Alexandru AND Sebastian Proksch AND Pooyan Behnamghader AND Harald C. Gall}, booktitle = {Working Conference on Software Visualization}, year = 2019, } -
"When Code Completion Fails: a Case Study on Real-World Completions".
Vincent Hellendoorn,
Sebastian Proksch,
Harald C. Gall,
Alberto Bacchelli.
International Conference on Software Engineering.
2019.
...
Code completion is commonly used by software developers and is integrated into all major IDE’s. Good completion tools can not only save time and effort but may also help avoid incorrect API usage. Many proposed completion tools have shown promising results on synthetic benchmarks, but these benchmarks make no claims about the realism of the completions they test. This lack of grounding in real-world data could hinder our scientific understanding of developer needs and of the efficacy of completion models. This paper presents a case study on 15,000 code completions that were applied by 66 real developers, which we study and contrast with artificial completions to inform future research and tools in this area. We find that synthetic benchmarks misrepresent many aspects of real-world completions; tested completion tools were far less accurate on real-world data. Worse, on the few completions that consumed most of the developers’ time, prediction accuracy was less than 20% – an effect that is invisible in synthetic benchmarks. Our findings have ramifications for future benchmarks, tool design and real-world efficacy: Benchmarks must account for completions that developers use most, such as intra-project APIs; models should be designed to be amenable to intra-project data; and real-world developer trials are essential to quantifying performance on the least predictable completions, which are both most time-consuming and far more typical than artificial data suggests. We publicly release our preprint and replication data and materials.
@inproceedings{HPGB+19-icse, title = {{When Code Completion Fails: a Case Study on Real-World Completions}}, author = {Vincent Hellendoorn AND Sebastian Proksch AND Harald C. Gall AND Alberto Bacchelli}, booktitle = {International Conference on Software Engineering}, year = 2019, } -
"Automated Reporting of Anti-Patterns and Decay in Continuous Integration".
Carmine Vassallo,
Sebastian Proksch,
Harald C. Gall,
Massimiliano Di Penta.
International Conference on Software Engineering.
2019.
...
Continuous Integration (CI) is a widely-used software engineering practice. The software is continuously built so that changes can be easily integrated and issues such as unmet quality goals or style inconsistencies get detected early. Unfortunately, it is not only hard to introduce CI into an existing project, but it is also challenging to live up to the CI principles when facing tough deadlines or business decisions. Previous work has identified common anti-patterns that reduce the promised benefits of CI. Typically, these anti-patterns slowly creep into a project over time before they are identified. We argue that automated detection can help with early identification and prevent such a process decay. In this work, we further analyze this assumption and survey 124 developers about CI anti-patterns. From the results, we build CI-Odor, a reporting tool for CI processes that detects the existence of four relevant anti-patterns by analyzing regular build logs and repository information. In a study on the 18,474 build logs of 36 popular Java projects, we reveal the presence of 3,823 high-severity warnings spread across projects. We validate our reports in a survey among 13 original developers of these projects and through general feedback from 42 developers that confirm the relevance of our reports.
@inproceedings{VPGD+19-icse, title = {{Automated Reporting of Anti-Patterns and Decay in Continuous Integration}}, author = {Carmine Vassallo AND Sebastian Proksch AND Harald C. Gall AND Massimiliano Di Penta}, booktitle = {International Conference on Software Engineering}, year = 2019, } -
"How Developers Engage with Static Analysis Tools in Different Contexts".
Carmine Vassallo,
Sebastiano Panichella,
Fabio Palomba,
Sebastian Proksch,
Harald C. Gall,
Andy Zaidman.
Empirical Software Engineering.
2019.
...
Automatic static analysis tools (ASATs) are instruments that support code quality assessment by automatically detecting defects and design issues. Despite their popularity, they are characterized by (i) a high false positive rate and (ii) the low comprehensibility of the generated warnings. However, no prior studies have investigated the usage of ASATs in different development contexts (e.g., code reviews, regular development), nor how open source projects integrate ASATs into their workflows. These perspectives are paramount to improve the prioritization of the identified warnings. To shed light on the actual ASATs usage practices, in this paper we first survey 56 developers (66% from industry and 34% from open source projects) and interview 11 industrial experts leveraging ASATs in their workflow with the aim of understanding how they use ASATs in different contexts. Furthermore, to investigate how ASATs are being used in the workflows of open source projects, we manually inspect the contribution guidelines of 176 open-source systems and extract the ASATs’ configuration and build files from their corresponding GitHub repositories. Our study highlights that (i) 71% of developers do pay attention to different warning categories depending on the development context; (ii) 63% of our respondents rely on specific factors (e.g., team policies and composition) when prioritizing warnings to fix during their programming; and (iii) 66% of the projects define how to use specific ASATs, but only 37% enforce their usage for new contributions. The perceived relevance of ASATs varies between different projects and domains, which is a sign that ASATs use is still not a common practice. In conclusion, this study confirms previous findings on the unwillingness of developers to configure ASATs and it emphasizes the necessity to improve existing strategies for the selection and prioritization of ASATs warnings that are shown to developers.
@inproceedings{VPPP+19-emse, title = {{How Developers Engage with Static Analysis Tools in Different Contexts}}, author = {Carmine Vassallo AND Sebastiano Panichella AND Fabio Palomba AND Sebastian Proksch AND Harald C. Gall AND Andy Zaidman}, booktitle = {Empirical Software Engineering}, year = 2019, }
2018
-
"On the Usage of Pythonic Idioms".
Carol Alexandru,
José J. Merchante,
Sebastiano Panichella,
Sebastian Proksch,
Harald C. Gall,
Gregorio Robles.
International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software, Onward!.
2018.
...
Developers discuss software architecture and concrete source code implementations on a regular basis, be it on question-answering sites, online chats, mailing lists or face to face. In many cases, there is more than one way of solving a programming task. Which way is best may be decided based on case-specific circumstances and constraints, but also based on convention. Having strong conventions, and a common vocabulary to express them, simplifies communication and strengthens common understanding of software development problems and their solutions. While many programming ecosystems have a common vocabulary, Python’s relationship to conventions and common language is a particularly pronounced. The “Zen of Python”, a famous set of high-level coding conventions authored by Tim Peters, states “There should be one, and preferably only one, obvious way to do it”. This ‘one way to do it’ is often referred to as the ‘Pythonic’ way: the ideal solution to a particular problem. Few other programming languages have coined a unique term to label the quality of craftsmanship gone into a software artifact. In this paper, we explore how Python developers understand the term ‘Pythonic’ by means of structured interviews, build a catalogue of ‘pythonic idioms’ gathered from literature, and conjecture on the effects of having a language-specific term for quality code, considering the potential it could hold for other programming languages and ecosystems. We find that while the term means different things to novice versus experienced Python developers, it encompasses not only concrete implementation, but a way of thinking – a culture – in general.
@inproceedings{AMPP+18-onward, title = {{On the Usage of Pythonic Idioms}}, author = {Carol Alexandru AND José J. Merchante AND Sebastiano Panichella AND Sebastian Proksch AND Harald C. Gall AND Gregorio Robles}, booktitle = {International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software, Onward!}, year = 2018, } -
"Redundancy-free Analysis of Multi-revision Software Artifacts".
Carol Alexandru,
Sebastiano Panichella,
Sebastian Proksch,
Harald C. Gall.
Empirical Software Engineering.
2018.
...
Researchers often analyze several revisions of a software project to obtain historical data about its evolution. For example, they statically analyze the source code and monitor the evolution of certain metrics over multiple revisions. The time and resource requirements for running these analyses often make it necessary to limit the number of analyzed revisions, e.g., by only selecting major revisions or by using a coarse-grained sampling strategy, which could remove significant details of the evolution. Most existing analysis techniques are not designed for the analysis of multi-revision artifacts and they treat each revision individually. However, the actual difference between two subsequent revisions is typically very small. Thus, tools tailored for the analysis of multiple revisions should only analyze these differences, thereby preventing re-computation and storage of redundant data, improving scalability and enabling the study of a larger number of revisions. In this work, we propose the “Lean Language-Independent Software Analyzer” (LISA), a generic framework for representing and analyzing multi-revisioned software artifacts. It employs a redundancy-free, multi-revision representation for artifacts and avoids re-computation by only analyzing changed artifact fragments across thousands of revisions. The evaluation of our approach consists of measuring the effect of each individual technique incorporated, an in-depth study of LISA’s resource requirements and a large-scale analysis over 7 million program revisions of 4,000 software projects written in four languages. We show that the time and space requirements for multi-revision analyses can be reduced by multiple orders of magnitude, when compared to traditional, sequential approaches.
@inproceedings{APPG18-emse, title = {{Redundancy-free Analysis of Multi-revision Software Artifacts}}, author = {Carol Alexandru AND Sebastiano Panichella AND Sebastian Proksch AND Harald C. Gall}, booktitle = {Empirical Software Engineering}, year = 2018, } -
"Investigating Order Information in API-Usage Patterns: A Benchmark and Empirical Study".
Ervina Cergani,
Sebastian Proksch,
Sarah Nadi,
Mira Mezini.
International Conference on Software Technologies.
2018.
...
Many approaches have been proposed for learning Application Programming Interface (API) usage patterns from code repositories. Depending on the underlying technique, the mined patterns may (1) be strictly sequential, (2) consider partial order between method calls, or (3) not consider order information. Understanding the trade-offs between these pattern types with respect to real code is important in many applications (e.g. code recommendation or misuse detection). In this work, we present a benchmark consisting of an episode mining algorithm that can be configured to learn all three types of patterns mentioned above. Running our benchmark on an existing dataset of 360 C# code repositories, we empirically study the resulting API usage patterns per pattern type. Our results show practical evidence that not only do partial-order patterns represent a generalized super set of sequential-order patterns, partial-order mining also finds additional patterns missed by sequence mining, which are used by a larger number of developers across code repositories. Additionally, our study empirically quantifies the importance of the order information encoded in sequential and partial-order patterns for representing correct co-occurrences of code elements in real code. Furthermore, our benchmark can be used by other researchers to explore additional properties of API patterns.
@inproceedings{CPNM18-icsoft, title = {{Investigating Order Information in API-Usage Patterns: A Benchmark and Empirical Study}}, author = {Ervina Cergani AND Sebastian Proksch AND Sarah Nadi AND Mira Mezini}, booktitle = {International Conference on Software Technologies}, year = 2018, } -
"Data-Driven Decisions and Actions in Today’s Software Development".
Harald C. Gall,
Carol Alexandru,
Adelina Ciurumelea,
Giovanni Grano,
Christoph Laaber,
Sebastiano Panichella,
Sebastian Proksch,
Gerald Schermann,
Carmine Vassallo,
Jitong Zhao.
The Essence of Software Engineering, Springer.
2018.
...
Today’s software development is all about data: data about the software product itself, about the process and its different stages, about the customers and markets, about the development, the testing, the integration, the deployment, or the runtime aspects in the cloud. We use static and dynamic data of various kinds and quantities to analyze market feedback, feature impact, code quality, architectural design alternatives, or effects of performance optimizations. Development environments are no longer limited to IDEs in a desktop application or the like but span the Internet using live programming environments such as Cloud9 or large-volume repositories such as BitBucket, Github, Gitlab, or StackOverflow. Software development has become “live” in the cloud, be it the coding, the testing, or the experimentation with different product options on the internet. The inherent complexity puts a a further burden on developers, since they need to stay alert when constantly switching between tasks in different phases. Research has been analyzing the development process, its data and stakeholders, for decades and is working on various tools that can help developers in their daily tasks to improve the quality of their work and their productivity. In this chapter, we critically reflect on the challenges faced by developers in a typical release cycle, identify inherent problems of the individual phases and present the current state of the research that can help overcoming these issues.
@incollection{GACG+18-ese, title = {{Data-Driven Decisions and Actions in Today’s Software Development}}, author = {Harald C. Gall AND Carol Alexandru AND Adelina Ciurumelea AND Giovanni Grano AND Christoph Laaber AND Sebastiano Panichella AND Sebastian Proksch AND Gerald Schermann AND Carmine Vassallo AND Jitong Zhao}, booktitle = {The Essence of Software Engineering}, publisher = Springer, year = 2018, } -
"Enriched Event Streams: A General Dataset For Empirical Studies On In-IDE Activities Of Software Developers".
Sebastian Proksch,
Sven Amann,
Sarah Nadi.
International Conference on Mining Software Repositories (Accepted Mining Challenge).
2018.
...
Developers have been the subject of many empirical studies over the years. To assist developers in their everyday work, an understanding of their activities is necessary, especially how they develop source code. Unfortunately, conducting such studies is very expensive and research often resorts to studying artifacts after the fact. To pave the road for future empirical studies on developer activities, we built FeedBaG, a general-purpose interaction tracker for Visual Studio that monitors development activities. The observations are stored in enriched event streams that encode a holistic picture on the in-IDE development process. Enriched event streams capture all commands invoked in the IDE with additional context information such as the test being run or the accompanying ne-grained code edits. We have used FeedBaG to collect enriched event streams from 81 developers. days. Over 1,527 days, we have collected more than 11M events that correspond to 15K hours of work time.
@inproceedings{PAN18-msr, title = {{Enriched Event Streams: A General Dataset For Empirical Studies On In-IDE Activities Of Software Developers}}, author = {Sebastian Proksch AND Sven Amann AND Sarah Nadi}, booktitle = {International Conference on Mining Software Repositories (Accepted Mining Challenge)}, year = 2018, } -
"Context Is King: The Developer Perspective on the Usage of Static Analysis Tools".
Carmine Vassallo,
Sebastiano Panichella,
Fabio Palomba,
Sebastian Proksch,
Andy Zaidman,
Harald C. Gall.
International Conference on Software Analysis, Evolution, and Reengineering.
2018.
...
Automatic static analysis tools (ASATs) are tools that support automatic code quality evaluation of software systems with the aim of (i) avoiding and/or removing bugs and (ii) spotting design issues. Hindering their wide-spread acceptance are their (i) high false positive rates and (ii) low comprehensibility of the generated warnings. Researchers and ASATs vendors have proposed solutions to prioritize such warnings with the aim of guiding developers toward the most severe ones. However, none of the proposed solutions considers the development context in which an ASAT is being used to further improve the selection of relevant warnings. To shed light on the impact of such contexts on the warnings configuration, usage and adopted prioritization strategies, we surveyed 42 developers (69% in industry and 31% in open source projects) and interviewed 11 industrial experts that integrate ASATs in their workflow. While we can confirm previous findings on the reluctance of developers to configure ASATs, our study highlights that (i) 71% of developers do pay attention to different warning categories depending on the development context, and (ii) 63% of our respondents rely on specific factors (e.g., team policies and composition) when prioritizing warnings to fix during their programming. Our results clearly indicate ways to better assist developers by improving existing warning selection and prioritization strategies.
@inproceedings{VPPP+18-saner, title = {{Context Is King: The Developer Perspective on the Usage of Static Analysis Tools}}, author = {Carmine Vassallo AND Sebastiano Panichella AND Fabio Palomba AND Sebastian Proksch AND Andy Zaidman AND Harald C. Gall}, booktitle = {International Conference on Software Analysis, Evolution, and Reengineering}, year = 2018, } -
"Un-Break My Build: Assisting Developers with Build Repair Hints".
Carmine Vassallo,
Sebastian Proksch,
Timothy Zemp,
Harald C. Gall.
International Conference on Program Comprehension.
2018.
...
Continuous integration is an agile software development practice. Instead of integrating features right before a release, they are constantly being integrated in an automated build process. This shortens the release cycle, improves software quality, and reduces time to market. However, the whole process will come to a halt when a commit breaks the build, which can happen for several reasons, e.g., compilation errors or test failures, and fixing the build suddenly becomes a top priority. Developers not only have to find the cause of the build break and fix it, but they have to be quick in all of it to avoid a delay for others. Unfortunately, these steps require deep knowledge and are often time consuming. To support developers in fixing a build break, we propose Bart, a tool that summarizes the reasons of the build failure and suggests possible solutions found on the Internet. We will show in a case study with eight participants that developers find Bart useful to understand build breaks and that using Bart substantially reduces the time to fix a build break, on average by 41%.
@inproceedings{VPZG18-icpc, title = {{Un-Break My Build: Assisting Developers with Build Repair Hints}}, author = {Carmine Vassallo AND Sebastian Proksch AND Timothy Zemp AND Harald C. Gall}, booktitle = {International Conference on Program Comprehension}, year = 2018, }
2017
-
"Developer Testing in the IDE: Patterns, Beliefs, and Behavior".
Moritz Beller,
Georgios Gousios,
Annibale Panichella,
Sebastian Proksch,
Sven Amann,
Andy Zaidman.
Transactions on Software Engineering.
2017.
...
Software testing is one of the key activities to achieve software quality in practice. Despite its importance, however, we have a remarkable lack of knowledge on how developers test in real-world projects. In this paper, we report on a large-scale field study with 2,443 software engineers whose development activities we closely monitored over 2.5 years in four integrated development environments (IDEs). Our findings, which largely generalized across the studied IDEs and programming languages Java and C#, question several commonly shared assumptions and beliefs about developer testing: half of the developers in our study do not test; developers rarely run their tests in the IDE; most programming sessions end without any test execution; only once they start testing, do they do it extensively; a quarter of test cases is responsible for three quarters of all test failures; 12% of tests show flaky behavior; Test-Driven Development (TDD) is not widely practiced; and software developers only spend a quarter of their time engineering tests, whereas they think they test half of their time. We summarize these practices of loosely guiding one’s development efforts with the help of testing in an initial summary on Test-Guided Development (TGD), a behavior we argue to be closer to the development reality of most developers than TDD.
@inproceedings{BGPP+17-tse, title = {{Developer Testing in the IDE: Patterns, Beliefs, and Behavior}}, author = {Moritz Beller AND Georgios Gousios AND Annibale Panichella AND Sebastian Proksch AND Sven Amann AND Andy Zaidman}, booktitle = {Transactions on Software Engineering}, year = 2017, } -
"Enriching In-IDE Process Information with Fine-Grained Source Code History".
Sebastian Proksch,
Sarah Nadi,
Sven Amann,
Mira Mezini.
International Conference on Software Analysis, Evolution, and Reengineering.
2017.
...
Current studies on software development either focus on the change history of source code from version-control systems or on an analysis of simplistic in-IDE events without context information. Each of these approaches contains valuable information that is unavailable in the other case. Our work proposes enriched event streams, a solution that combines the best of both worlds and provides a holistic view on the software development process. Enriched event streams not only capture developer activities in the IDE, but also specialized context information, such as source-code snapshots for change events. To enable the storage of such code snapshots in an analyzable format, we introduce a new intermediate representation called Simplified Syntax Trees (SSTs) and build CARET, a platform that offers reusable components to conveniently work with enriched event streams. We implement FeedBaG++, an instrumentation for Visual Studio that collects enriched event streams with code snapshots in the form of SSTs. We share a dataset of enriched event streams captured from 58 users and representing 915 days of work. Additionally, to demonstrate usefulness, we present three research applications that have already made use of CARET and FeedBaG++.
@inproceedings{PNAM17-saner, title = {{Enriching In-IDE Process Information with Fine-Grained Source Code History}}, author = {Sebastian Proksch AND Sarah Nadi AND Sven Amann AND Mira Mezini}, booktitle = {International Conference on Software Analysis, Evolution, and Reengineering}, year = 2017, } -
"Enriched Event Streams: A General Platform For Empirical Studies On In-IDE Activities Of Software Developers".
Sebastian Proksch.
PhD Thesis,
Technische Universität Darmstadt, Germany.
2017.
...
Current studies on software development either focus on the change history of source code from version-control systems or on an analysis of simplistic in-IDE events without context information. Each of these approaches contains valuable information that is unavailable in the other case. This work proposes enriched event streams, a solution that combines the best of both worlds and provides a holistic view on the in-IDE software development process. Enriched event streams not only capture developer activities in the IDE, but also specialized context information, such as source-code snapshots for change events. To enable the storage of such code snapshots in an analyzable format, we introduce a new intermediate representation called Simplified Syntax Trees (SSTs) and build CARET, a platform that offers reusable components to conveniently work with enriched event streams. We implement FeedBaG++, an instrumentation for Visual Studio that collects enriched event streams with code snapshots in the form of SSTs and share a dataset of enriched event streams captured in an ongoing field study from 81 users and representing 15K hours of active development. We complement this with a dataset of 69M lines of released source code extracted from 360 GitHub repositories. To demonstrate the usefulness of our platform, we use it to conduct studies on the in-IDE development process that are both concerned with source-code evolution and the analysis of developer interactions. In addition, we build recommendation systems for software engineering and analyze and improve current evaluation techniques.
@inproceedings{Pro17-thesis, title = {{Enriched Event Streams: A General Platform For Empirical Studies On In-IDE Activities Of Software Developers}}, author = {Sebastian Proksch}, booktitle = {PhD Thesis, Technische Universität Darmstadt, Germany}, year = 2017, } -
"On the Positive Effect of Reactive Programming on Software Comprehension: An Empirical Study".
Guido Salvaneschi,
Sebastian Proksch,
Sven Amann,
Sarah Nadi,
Mira Mezini.
Transactions on Software Engineering.
2017.
...
Starting from the first investigations with strictly functional languages, reactive programming has been proposed as the programming paradigm for reactive applications. Over the years, researchers have enriched reactive languages with more powerful abstractions, embedded these abstractions into mainstream languages – including object-oriented languages – and applied reactive programming to several domains, like GUIs, animations, Web applications, robotics, and sensor networks. However, an important assumption behind this line of research is that, beside other claimed advantages, reactive programming makes a wide class of otherwise cumbersome applications more comprehensible. This claim has never been evaluated. In this paper, we present the first empirical study that evaluates the effect of reactive programming on comprehension. The study involves 127 subjects and compares reactive programming to the traditional object-oriented style with the Observer design pattern. Our findings show that program comprehension is significantly enhanced by the reactive-programming paradigm – a result that suggests to further develop research in this field.
@inproceedings{SPANM17-tse, title = {{On the Positive Effect of Reactive Programming on Software Comprehension: An Empirical Study}}, author = {Guido Salvaneschi AND Sebastian Proksch AND Sven Amann AND Sarah Nadi AND Mira Mezini}, booktitle = {Transactions on Software Engineering}, year = 2017, }
2016
-
"FeedBaG: An Interaction Tracker for Visual Studio".
Sven Amann,
Sebastian Proksch,
Sarah Nadi.
International Conference on Program Comprehension.
2016.
...
Integrated Development Environments (IDEs) provide a convenient standalone solution that supports developers during various phases of software development. In order to provide better support for developers within such IDEs, we need to understand how developers use them. To infer useful conclusions, such information should be gathered for different types of IDEs, for different programming languages, and in different development settings.
In this paper, we present FEEDBAG, an extension for Visual Studio that tracks developers’ interactions with the IDE. FEEDBAG generates a rich stream of interaction events and provides means for developers to review and submit the data to a server. We recently used the tool in a study, recording more than 6,300 hours of work time. Future studies with different user groups are needed to explore and compare IDE-usage aspects, like code-comprehension assistance, in detail. Therefore, we publish FEEDBAG and encourage other researchers to use it as well.
@inproceedings{APN16-icpc, title = {{FeedBaG: An Interaction Tracker for Visual Studio}}, author = {Sven Amann AND Sebastian Proksch AND Sarah Nadi}, booktitle = {International Conference on Program Comprehension}, year = 2016, } -
"A Study of Visual Studio Usage in Practice".
Sven Amann,
Sebastian Proksch,
Sarah Nadi,
Mira Mezini.
International Conference on Software Analysis, Evolution, and Reengineering.
2016.
...
Integrated Development Environments (IDEs) provide a convenient standalone solution that supports developers during various phases of software development. In order to provide better support for developers within such IDEs, we need to understand how much time developers spend using various parts of a given IDE and how often they use available assistance tools. To infer useful conclusions, such information should be gathered for different types of IDEs for different languages. In this paper, we instrument the previously unexplored Visual Studio IDE and track the interactions of developers at an industry partner’s software-development department. As a result, we capture interactions for more than 6300 hours of work time, from between 27 and 84 professional C# developers. Our work reports how much time professional developers spend on activities such as code editing and execution or navigation, as well as how often they use assistance tools provided by the IDE. We compare our findings to those of prior studies involving other IDEs and discuss the implications of the commonalities and differences for research on (integrated) developer-assistance tools.
@inproceedings{APNM16-saner, title = {{A Study of Visual Studio Usage in Practice}}, author = {Sven Amann AND Sebastian Proksch AND Sarah Nadi AND Mira Mezini}, booktitle = {International Conference on Software Analysis, Evolution, and Reengineering}, year = 2016, } -
"Addressing Scalability in API Method-Call Analytics".
Ervina Cergani,
Sebastian Proksch,
Sarah Nadi,
Mira Mezini.
International Workshop on Software Analytics.
2016.
...
Intelligent code completion recommends relevant code to developers by comparing the editor content to code patterns extracted by analyzing large repositories. However, with the vast amount of data available in such repositories, scalability of the recommender system becomes an issue. We propose using Boolean Matrix Factorization (BMF) as a clustering technique for analyzing code in order to improve scalability of the underlying models. We compare model size, inference speed, and prediction quality of an intelligent method call completion engine built on top of canopy clustering versus one built on top of BMF. Our results show that BMF reduces model size up to 80% and increases inference speed up to 78%, without significant change in prediction quality.
@inproceedings{CPNM16-swan, title = {{Addressing Scalability in API Method-Call Analytics}}, author = {Ervina Cergani AND Sebastian Proksch AND Sarah Nadi AND Mira Mezini}, booktitle = {International Workshop on Software Analytics}, year = 2016, } -
"Evaluating the Evaluations of Code-Recommender Systems: A Reality Check".
Sebastian Proksch,
Sven Amann,
Sarah Nadi,
Mira Mezini.
International Conference on Automated Software Engineering.
2016.
...
While researchers develop many new exciting code recommender systems, such as method-call completion, code-snippet completion, or code search, an accurate evaluation of such systems is always a challenge. We analyzed the current literature and found that most of the current evaluations rely on artificial queries extracted from released code, which begs the question: Do such evaluations reflect real-life usages? To answer this question, we capture 6,189 fine-grained development histories from real IDE interactions. We use them as a ground truth and extract 7,157 real queries for a specific method-call recommender system. We compare the results of such real queries with different artificial evaluation strategies and check several assumptions that are repeatedly used in research, but never empirically evaluated. We find that an evolving context that is often observed in practice has a major effect on the prediction quality of recommender systems, but is not commonly reflected in artificial evaluations.
@inproceedings{PANM16-ase, title = {{Evaluating the Evaluations of Code-Recommender Systems: A Reality Check}}, author = {Sebastian Proksch AND Sven Amann AND Sarah Nadi AND Mira Mezini}, booktitle = {International Conference on Automated Software Engineering}, year = 2016, } -
"A Dataset of Simplified Syntax Trees for C#".
Sebastian Proksch,
Sven Amann,
Sarah Nadi,
Mira Mezini.
International Conference on Mining Software Repositories.
2016.
...
In this paper, we present a curated collection of 2833 C# solutions taken from Github. We encode the data in a new intermediate representation (IR) that facilitates further analysis by restricting the complexity of the syntax tree and by avoiding implicit information. The dataset is intended as a standardized input for research on recommendation systems for software engineering, but is also useful in many other areas that analyze source code.
@inproceedings{PANM16-msr, title = {{A Dataset of Simplified Syntax Trees for C#}}, author = {Sebastian Proksch AND Sven Amann AND Sarah Nadi AND Mira Mezini}, booktitle = {International Conference on Mining Software Repositories}, year = 2016, }
2015
-
"How to Build a Recommendation System for Software Engineering".
Sebastian Proksch,
Veronika Bauer,
Gail C. Murphy.
Software Engineering, Springer.
2015.
...
Software developers must interact with large amounts of different types of information and perform many different activities to build a software system. To ease the finding of information and hone workflows, there has been growing interest in building recommenders that are intended to help software developers work more effectively. Building an effective recommender requires a deep understanding of the problem that is the target of a recommender, analysis of different aspects of the approach taken to perform the recommendations and design and evaluation of the mechanisms used to present recommendations to a developer. In this chapter, we outline the different steps that must be taken to develop an effective recommender system to aid software development.
@incollection{PBM15-laser, title = {{How to Build a Recommendation System for Software Engineering}}, author = {Sebastian Proksch AND Veronika Bauer AND Gail C. Murphy}, booktitle = {Software Engineering}, publisher = Springer, year = 2015, } -
"Intelligent Code Completion with Bayesian Networks".
Sebastian Proksch,
Johannes Lerch,
Mira Mezini.
Transactions on Software Engineering and Methodology.
2015.
...
Code completion is an integral part of modern Integrated Development Environments (IDEs). Developers often use it to explore Application Programming Interfaces (APIs). It is also useful to reduce the required amount of typing and to help avoid typos. Traditional code completion systems propose all type-correct methods to the developer. Such a list is often very long with many irrelevant items. More intelligent code completion systems have been proposed in prior work to reduce the list of proposed methods to relevant items.
This work extends one of these existing approaches, the Best Matching Neighbor (BMN) algorithm. We introduce Bayesian networks as an alternative underlying model, use additional context information for more precise recommendations, and apply clustering techniques to improve model sizes. We compare our new approach, Pattern-based Bayesian Networks (PBN), to the existing BMN algorithm. We extend previously used evaluation methodologies and, in addition to prediction quality, we also evaluate model size and inference speed.
Our results show that the additional context information we collect improves prediction quality, especially for queries that do not contain method calls. We also show that PBN can obtain comparable prediction quality to BMN, while model size and inference speed scale better with large input sizes.
@inproceedings{PLM15-tosem, title = {{Intelligent Code Completion with Bayesian Networks}}, author = {Sebastian Proksch AND Johannes Lerch AND Mira Mezini}, booktitle = {Transactions on Software Engineering and Methodology}, year = 2015, }
2014
-
"Method-Call Recommendations from Implicit Developer Feedback".
Sven Amann,
Sebastian Proksch,
Mira Mezini.
International Workshop on CrowdSourcing in Software Engineering.
2014.
...
When developers use the code completion in their Integrated Development Environment (IDE), they provide implicit feedback about the usage of the Application Programming Interfaces (APIs) they program against. We demonstrate how to apply Collaborative Filtering techniques to compute context-sensitive completion recommendations from such feedback and discuss how the approach can be used to bring the knowledge of the crowd to every developer.
@inproceedings{APM14-csise, title = {{Method-Call Recommendations from Implicit Developer Feedback}}, author = {Sven Amann AND Sebastian Proksch AND Mira Mezini}, booktitle = {International Workshop on CrowdSourcing in Software Engineering}, year = 2014, } -
"Towards Standardized Evaluation of Developer-Assistance Tools".
Sebastian Proksch,
Sven Amann,
Mira Mezini.
International Workshop on Recommendation Systems for Software Engineering.
2014.
...
Over the last years, researchers proposed a variety of assistance tools to support developers in their development environments. Many of the respective publications introduce new evaluation strategies or use custom datasets. Size and quality of the performed evaluations differ. Additionally, the strategies often use metrics that are tailored to the respective tools. As a result, comparing different assistance tools is very difficult.
In this work, we present a framework for the standardized evaluation of assistance tools, on the example of code recommenders. The framework combines different ideas and demands from previous work. Furthermore, we discuss how the community could jointly realize the framework.
@inproceedings{PAM14-rsse, title = {{Towards Standardized Evaluation of Developer-Assistance Tools}}, author = {Sebastian Proksch AND Sven Amann AND Mira Mezini}, booktitle = {International Workshop on Recommendation Systems for Software Engineering}, year = 2014, } -
"An empirical study on program comprehension with reactive programming".
Guido Salvaneschi,
Sven Amann,
Sebastian Proksch,
Mira Mezini.
International Symposium on Foundations of Software Engineering.
2014.
...
Starting from the first investigations with strictly functional languages, reactive programming has been proposed as THE programming paradigm for reactive applications. The advantages of designs based on this style over designs based on the Observer design pattern have been studied for a long time. Over the years, researchers have enriched reactive languages with more powerful abstractions, embedded these abstractions into mainstream languages – including object-oriented languages – and applied reactive programming to several domains, like GUIs, animations, Web applications, robotics, and sensor networks. However, an important assumption behind this line of research – that, beside other advantages, reactive programming makes a wide class of otherwise cumbersome applications more comprehensible – has never been evaluated. In this paper, we present the design and the results of the first empirical study that evaluates the effect of reactive programming on comprehensibility compared to the traditional object-oriented style with the Observer design pattern. Results confirm the conjecture that comprehensibility is enhanced by reactive programming. In the experiment, the reactive programming group significantly outperforms the other group.
@inproceedings{SAPM14-fse, title = {{An empirical study on program comprehension with reactive programming}}, author = {Guido Salvaneschi AND Sven Amann AND Sebastian Proksch AND Mira Mezini}, booktitle = {International Symposium on Foundations of Software Engineering}, year = 2014, }
2011
-
"Empirical Study of Impact Factors on the Quality of Code-Recommender Models".
Sebastian Proksch.
Master Thesis,
Technische Universität Darmstadt, Germany.
2011.
...
Code recommender systems exist that integrate into the development environment to support a developer by learning and understanding new frameworks. The Eclipse Code Recommenders project provides several of such systems. One of them is an engine for intelligent code completion that can be used to get suggestions based on the currently written source code. This engine is named calls recommender because it can be used to query missing method calls for an object instance.
To be able to propose methods, the usage of various frameworks is learned by an analysis of a large repository of open source projects. The gained knowledge is re-integrated into the popular Java editor Eclipse through a plugin that enhances the default code completion. It recommends methods that might be interesting for the developer in the current context. The proposal is based on the source code that was already written in the editor.
The motivation of this work is to further improve the existing recommender by focussing on three objectives: (i) The models of the recommender should make use of additional features that describe the definition of an object instance in the source code to further increase the predictive power. (ii) The learning algorithm should be replaced by a more sophisticated approach that also makes use of these new features. (iii) Additionally, the memory usage of the recommender models should be reduced.
All these objectives influence each other and it is necessary to evaluate the impact on the predictive power of the code recommender models. It is shown in this work that all objectives can be achieved in combination. Several modifications for the calls recommender are introduced and implemented to be able to replace the previous recommender.
@inproceedings{Pro11-msc, title = {{Empirical Study of Impact Factors on the Quality of Code-Recommender Models}}, author = {Sebastian Proksch}, booktitle = {Master Thesis, Technische Universität Darmstadt, Germany}, year = 2011, }
2009
-
"Annotation-Support for Generating Animations of Algorithms".
Sebastian Proksch.
Bachelor Thesis,
Technische Universität Darmstadt, Germany.
2009.
...
Understanding algorithms is daily work for computer scientists. An adequate visualization is a useful aid for this learning process. Especially teachers often have the problem that building an animation from scratch is too much work and that existing animations, tools or programming framework are too unflexible to cover all their needs.
This thesis starts at this point. A concept will be developed, to be able to enrich a visualization based on annotations as well as to implement own programm logic based on annotations. This possibility should be integrated into Animal, to soften the border between programming inside the framework and programming based on the annotations. This approach tries to enable easy access to the framework also for unexperienced programmers and in addition permits to extend existing animations in an easy way.
@inproceedings{Pro09-bsc, title = {{Annotation-Support for Generating Animations of Algorithms}}, author = {Sebastian Proksch}, booktitle = {Bachelor Thesis, Technische Universität Darmstadt, Germany}, year = 2009, }