Overview

This workshop is intended to encourage interdisciplinary research between NLP and Software Engineering resources. We invited a range of researchers with both NLP and SE backgrounds to come together, discuss their research, establish datasets, tasks, and baselines, and generally help the field build momentum.

Recent updates

New!
Please add suggsted resources and datasets to this spreadsheet. We'll review and prioritize in the closing session.

New!
Tuesday sessions were rearranged; please check the schedule.

Call for participation

Get the current call for participation as a PDF.

Submissions

Please submit your 1–2 page paper here:
https://www.easychair.org/conferences/?conf=nlse2015

Important dates

Sept 1, 2015

1-2 page paper due

Oct 1, 2015

Program finalized and sent out to attendees

Oct 25–27, 2015

Workshop dates

Dec 15, 2015

Final report produced

Attendee list

Workshop program

This three day workshop will be held Sunday, October 25 until Tuesday, October 27, 2015. All sessions will be the large lecture hall in Microsoft Building 99, room 99/1919.

Program overview

Sunday, October 25

1:00pm – 2:30pm

Tutorial session: n-gram and Neural Network Language Modeling

Ashish Vaswani

PDF slides

2:30pm – 3:00pm

coffee break

3:00pm – 4:30pm

Tutorial session: Software Mining and Software Datasets

Tao Xie

PDF slides

4:30pm – 5:30pm

break for check in, etc.

6:00pm – 8:30pm

Dinner, catered at Microsoft Building 99

Monday, October 26

8:00am – 9:00am

Light breakfast in 99/1919

9:00am – 10:15am

Software tools and processes

(Organizers: Premkumar Devanbu and Chris Quirk)

(Scribe: Jennifer D’Souza)

09h00 – 09h30 Opening session and introduction

09h30 – 10h15 Keynote by Charles Sutton

10:15am – 10:30am

Coffee break

10:30am – 11:15am

Software tools and processes (cont'd)

10h30 – 11h15 Open Discussion

1. What kind of collaborations would help move this area forward?
2. Are there other questions/problems that remain unexplored?
3. What data resources are needed?
4. Are there benchmarks or evaluation contests that are needed?

11:15am – 12:00pm

Data repositories

(Organizers: Premkumar Devanbu and Chris Quirk)

(Scribe: Zhilin Yang)

11h15 – 11h45 Talk by Tien Nguyen: The BOA Code Repository and Infrastructure

11h45 – 12h00 Questions/Discussion

12:00pm – 1:00pm

Lunch break

1:00pm – 2:15pm

Mutual introductions: Two minute madness

2:15pm – 2:30pm

Coffee break

2:30pm – 4:00pm

Code and Program Modeling

(Organizers: Charles Sutton and Tien Nguyen)

(Scribe: Vincent Hellendoorn)

2h30 – 3h00 Talk by Earl Barr, University College London: Inference Problems in Software Engineering

3h00 – 3h30 Talk by Daniel Tarlow, Microsoft Research

3h30 – 4h00 Open Discussion

1. Applications of code and program modeling?
2. When are different levels of information appropriate, e.g., lexical, syntactic, semantic?
3. Calling all hammers: What are NLP modeling techniques that are ripe for carrying over?
4. What’s special about software? Can we just keep importing standard NLP methods willy-nilly or do we need methods that are SWE-specific?

4:00pm – 4:15pm

Coffee break

4:15pm – 5:15pm

Ontologies and Understanding of Software Semantics

(Organizers: Dana Movshovitz-Attias and Tao Xie)

(Scribe: Zhilin Yang)

4:15pm – 4:35pm Talk by Jane Cleland-Huang: Leveraging Software Project Knowledge to Build Ontology

4:35pm – 5:15pm Open Discussion

1. What are software engineering tasks that can benefit from a software ontology or semantic understanding?
2. What are the unique characteristics of software entities that make them more/less susceptible for semantic analysis?
3. Emerging NLP techniques that can be leveraged for software semantic understanding. Which ones have been successfully used by the participants?
4. Open problems or challenges in this area.
5. Available software ontologies or related resources.
6. Evolving software ontologies. What changes can cause a software ontology to evolve over time? What are methods for accommodating such changes?
7. What kind of collaborations would help move this area forward?

5:15pm – 6:00pm

Free time

6:00pm – 9:00pm

Dinner at Luc in the Madison Park neighborhood of Seattle; transportation from Microsoft and to Microsoft+hotel provided

Tuesday, October 27

8:00am – 9:00am

Light breakfast in 99/1919

9:00am – 10:30am

Information Retrieval in Software Engineering

(Organizers: Denys Poshyvanyk and Dana Movshovitz-Attias)

(Scribe: Martin White)

09h00 – 09h20 Talk by Andrian Marcus: Overview of Text Retrieval Applications in Software Engineering

09h20 – 10h30 Open Discussion (sample questions are below)

Examples of successful ideas (applications of IR in SE): participants talk for 1 minute to give examples of their prior research projects

1. Open problems or grand challenges?
2. Emerging Information Retrieval techniques or approaches?
3. What kind of collaborations would help move this area forward?
4. What datasets and resources are available?
5. Challenges in reproducibility of the experiments

10:30am – 10:45am

Coffee break

10:45am – 12:15pm

Natural Language Programming and Semantic Parsing

(Organizers: Ray Mooney and Chris Quirk)

(Scribe: Gagan Bansal)

A set of 5 minute talks by:

1. Ray Mooney
2. Chris Quirk
3. Gagan Bansal
4. Yoav Artzi
5. Percy Liang
6. Srini Iyer
7. Nate Kushman
8. Yi Wei

Open forum discussion, seeded by particular topics:

• Data
• Compute cycles
• Techniques: Is it parsing? Is it MT? Is it program synthesis?
• Tools
• Evaluation metrics
• What are the relevant connections in SE?
• Dialog -- interaction strategies
• End-user debugging
• Where do we focus first? End users? Programmers? Power users
• Formalism / representation, especially for end users
• Adding I/O tuples or programming by demonstration

12:15pm – 1:15pm

Lunch break

1:15pm – 2:45pm

Language Generation from Code

(Organizers: Dawn Lawrie and Graham Neubig)

(Scribe: Vincent Hellendoorn)

Mood setting talk: Survey of Methods to Generate Natural Language from Source Code, Graham Neubig (Slides)

Each of the other researches in the topic will be asked to produce at controversial statement or question to fuel discussion. Discussion will also touch on data sets and broader impacts. Other researchers will be asked about two weeks before the workshop to think about their controversial statement.

2:45pm – 3:00pm

Coffee break

3:00pm – 4:30pm

Closing session

Venue

The workshop will be held at Microsoft Research in Redmond, WA.

The address is:

Directions to visit the venue are available here.

Accommodations

We have reserved a block of rooms at the Courtyard in Bellevue Redmond.

To book, please call (800) 321-2211 and ask for the Microsoft NL + SE Room Block Oct2015 at the Courtyard in Bellevue Redmond. Or you can use this online reservation link

Please book by October 5th to ensure that you receive the block rate.

The address is:

Travel

Support for travel and accommodation are offered through a grant from the National Science Foundation. Covered costs include hotel, accommodation and local transit. Our budget can cover travel expenses up to:

• USD 1900 for international attendees
• USD 1000 for US West Coast attendees
• USD 1300 for attendees from other parts of the US

Attendees from the Puget Sound area should contact organizers if they need travel support.

Reimbursement

1. Include paid receipts for airfare, hotel, airport shuttle, and meals.
2. Note: no alcohol can be reimbursed.
3. Use UC Davis Form here, sign as "Non Employee"
4. Maximum budgeted amount for academic invitees: $1000 for West coast, $1300 for domestic attendees, and $1900 for international.
5. Send to Jane Ryan, Dept of Computer Science, UC Davis, Davis, CA 95616; mark (Attn: NSF Workshop)
6. Be sure to include return postal address, full name, and telephone number.

Sponsors

The workshop is being sponsored by the US National Science Foundation through Sol Greenspan and Tatiana Korelsky and Microsoft Research.

Steering committee

• Charles Sutton (Edinburgh)
• Daniel Tarlow (Microsoft Research, Cambridge)
• Dawn Lawrie (Loyola)
• Dennis Poshyvanyk (Willam & Mary)
• Ray Mooney (University of Texas Austin)
• Tao Xie (University of Illinois)
• William Cohen (Carnegie Mellon University)

Organizers

Prem Devanbu (UC Davis) and Chris Quirk (Microsoft Research) along with Dana Movshovitz-Attias (CMU) organized the workshop, under the guidance and direction of Sol Greenspan and Tatiana Korelsky from the NSF.

Organizer information here