See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/335361278

Towards Effective AI-Powered Agile Project Management

Conference Paper · May 2019

DOI: 10.1109/ICSE-NIER.2019.00019

CITATIONS READS
50 1,711

5 authors, including:

Hoa Khanh Dam Truyen Tran

University of Wollongong Deakin University
145 PUBLICATIONS 2,507 CITATIONS 239 PUBLICATIONS 5,683 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Truyen Tran on 06 December 2019.

The user has requested enhancement of the downloaded file.

Towards effective AI-powered agile project
management
Hoa Khanh Dam Truyen Tran John Grundy
University of Wollongong, Australia Deakin University, Australia Monash University, Australia
[email protected] [email protected] [email protected]

Aditya Ghose Yasutaka Kamei

University of Wollongong, Australia Kyushu University, Japan
[email protected] [email protected]

Abstract—The rise of Artificial intelligence (AI) has the poten- An agile project are centered around a product backlog
tial to significantly transform the practice of project management. (see Figure 1), which is typically a collection of items to be
Project management has a large socio-technical element with completed in the project [3]. Items in a product backlog can be,
many uncertainties arising from variability in human aspects,
e.g. customers’ needs, developers’ performance and team dy- for example, customer requirements for the product (user sto-
namics. AI can assist project managers and team members ries), requests for bug fixes, changes to existing features, and
by automating repetitive, high-volume tasks to enable project technical improvements. Product backlog is evolved through
analytics for estimation and risk prediction, providing actionable regular updates and refinement to ensure that it contains items
recommendations, and even making decisions. AI is potentially that are relevant to project’s scope and objectives, sufficiently
a game changer for project management in helping to accelerate
productivity and increase project success rates. In this paper, we detailed, and appropriately estimated. Important updates to the
propose a framework where AI technologies can be leveraged to product backlog include adding or removing backlog items
offer support for managing agile projects, which have become based on current needs, estimating the size of items, and refine
increasingly popular in the industry. large items into small fine-grained items.
Index Terms—Software Engineering, Artificial Intelligence

Item Size
I. I NTRODUCTION 5 Estimate
8
Artificial Intelligence (AI) has started making a substantial Add new item

impact to many parts of our society, and is predicted to disrupt Reprioritize items
how we produce, manufacture, and deliver. The rise of AI Refine items
Task breakout
Sprint
is empowered by the growth and availability of big data, Remove items Sprint planning
Working
breakthroughs in AI algorithms (e.g. deep learning), and sig- Product backlog
product
Sprint backlog
nificantly increased computational power. The pervasiveness
of software products has resulted in a massive amount of data Fig. 1. A typical agile process
about software projects which AI techniques can leverage. We
envision that AI will transform (software) project management An agile project consists of multiple iterations (or alterna-
practice in many aspects, from automating basic administra- tively referred as sprints). Each sprint is often a short period
tion tasks to delivering analytics-driven risk predictions and in which the team aims to complete a subset of items in
estimation, facilitating project planning and making actionable the product backlog. Prior to a sprint, the team performs
recommendations. In this paper, we present a framework of sprint planning to identify the goal of that upcoming sprint,
how various AI technologies are adapted and integrated to and select items from the product backlog which they will
support various areas of agile project management (agile PM). complete to meet the sprint’s goal. During sprint planning,
Agile methods (e.g. Scrum) have been widely used in many agile teams decompose each product backlog items into
industry to manage software projects [1]. This relatively new a set of tasks. These tasks and their corresponding product
approach to project management empowers software teams backlog items form the sprint backlog. The team then executes
to focus on rapid delivery of business value to customers, the sprint to complete items in the sprint backlog to deliver a
thus significantly reducing the overall risk of project failures. potentially shippable product increment.
Project management has thus witnessed a shift away from the
traditional “waterfall” process and towards a more adaptive, II. C HALLENGES IN AGILE P ROJECT M ANAGEMENT
agile model. The number of projects following agile has Many tools have been developed to support agile project
increased significantly in the recent years, not only in the management such as JIRA, Assembla and Axosoft. Those
software industry but also in other non-IT domains [2]. tools allow agile teams to create and manage various agile
artifacts such as user stories, product backlogs, sprints, and must be considered, including items contributing toward the
sprint backlogs. For example, they support teams in creating sprint goal, their priority and business value to customers,
user stories and tasks, linking related tasks and user stories, the dependencies among items, appropriate allocations to bug
assigning team members to tasks and issues, creating dead- fixing and other technical work (e.g. resolving technical debts)
lines, setting priorities and estimates (e.g. story points). They and the availability of team members and the team’s capacity.
also enable team members to see the amount of work required Risks impeding a sprint execution should also be forecasted
for individuals and teams during each sprint, track progress and factored into a sprint plan. Sprint planning thus requires
of the sprint and its associated user stories and tasks. They not only in-depth understanding of the current project and
facilitate real-time information exchange and collaboration via team but also experience learned from previous projects. Tool
centralised project information. support is needed to manage complexities for large projects.
Although existing agile tools are useful, their support is 4) Pro-actively monitoring sprint progress and managing
limited to creating, managing, and tracking project artifacts, risks: As the sprint unfolds, the team needs to track sprint
and visualising historical project data such as burndown charts progress and manage risks. Current practices in risk man-
and other agile reports. Current agile project management agement mostly rely on high-level guidance and subjective
tools lack advanced analytical methods that are capable of judgements. Predicting future risks is highly challenging due
harvesting valuable insights from project data for predic- to the inherent uncertainty, temporal dependencies, and es-
tion, estimation, planning and action recommendation. Many pecially the dynamic nature of software. There is currently
decision-making tasks in agile projects are still performed by a gap in providing agile teams with insightful and actionable
agile teams without machinery support. We identify a number information about the current existence of risks in a sprint, and
of important areas in agile project management that remain recommending concrete measures to deal with those risks.
challenging due to this lack of effective support.
1) Identifying backlog items: Items in the product backlog III. A N AI–P OWERED AGILE P ROJECT A SSISTANT
can be derived from different sources such as a requirement The above challenges and the serious lack of effective tools
specification, new feature requests from customers, bugs re- presents an opportunity for AI to significantly improve the
ported by end users, previous bug fixes, discussions among practice of agile project management. AI-based tools are able
agile teams (e.g. technical debts, design changes or action to process massive amounts of data generated from software
items from retrospective meetings), end users’ reviews of projects, harvest useful insights, and train to perform complex
the product, and even experiences from previous projects. It tasks such as estimating effort, task refinement, resource
is difficult and time consuming for agile teams, especially management, and sprint planning. Figure 2 shows our proposal
product owners, to process this large amount of heterogenous for the architecture of an AI-powered agile project manage-
data in order to identify and create new items for the product ment assistant. The core of this AI system are an analytics
backlog. In addition, for each newly created backlog item, it engine, a planning engine and an optimization engine. These
is necessary to consider inter-dependencies between the new machineries depends on the representation learning engine
item and existing ones. This is challenging as a typical project to learn and generate representations of project data that are
has a large product backlog with more than 100 items. mathematically and computationally convenient to process.
2) Refining backlog items: Some items (e.g. user stories) The conversational dialog engine converses with users and
in the product backlog are initially large, thus do not fit within brings the support provided by the other engines to the users.
a single sprint. Agile teams are often required to refine these
large items into small ones such that they not only facilitate A. Representation learning engine
implementation but are sufficiently large, allowing stakehold- Agile project artifacts contain both structured and unstruc-
ers to understand business value [3]. There are typically three tured data. For example, backlog items may have structured
levels of refinement: (1) decomposing an epic into a number attributes such as type and priority (which are easily extracted
of user stories; (2) splitting user stories into small stories; and to form a vector representation), whereas product visions,
(3) breaking a user story into a number of specific project sprint goals, description of backlog items, and communication
tasks. Different rules and guidelines have been proposed to among team members (e.g. comments on backlog items) are
help teams refine backlog items, but rules often overlap with written in natural text. Codebases contain documentations such
or even conflict with one another. Teams struggle to refine as release notes and comments written in natural text, and
backlog items and rely on their own intuition and experience. source code written in programming languages. Hence, the
3) Sprint planning: The key part of sprint planning is representation learning engine is an important component of
selecting a subset of items in the backlog which can re- this AI system, responsible for learning vector representations
alistically be accomplished by the team in the upcoming for each project artifact. These representations can automati-
sprint to deliver a product increment. The customer expects cally be learned from unlabelled data, and are then used by
the team to deliver what have been planned for a sprint, the other machineries in the AI system.
thus meeting this expectation is important in maintaining the The representation learning engine has a NLP component
customer’s faith in the team’s ability to deliver. Sprint planning which performs automatic analysis on project textual artifacts
is however highly challenging since many important factors and then generates vector representations of those artifacts.
Product features of a sprints based on the features of the backlog items
backlog Agile team
items
Codebase assigned to it. We will extend those aggregation methods to
Product learn features for representing team members. This represen-
vision
Sprint tation will be enriched with features representing work and
goals
Representation
Sprints social dependencies between team members, extracted from

Conversational Dialog Engine

Sprint plan
Learning Engine communication logs (e.g. comments or discussions on work
NLP Code modeling items).
Feature Extraction
Planning Engine and Aggregation
Backlog item B. Analytics engine
identification
Reinforcement
deep learner The analytics engine aims to provide decision support in
Risk mitigation
Analytics Engine the following aspects:
Prescriptive Analytics Backlog item
refinement
1) Descriptive analytics: Most existing agile project man-
Optimization Engine
(e.g. Search-Based SE) Predictive Analytics agement tools support this basic level of analytics: data vi-
Risk prediction sualization via reports, dashboards, and scorecards. Common
Descriptive Analytics
Backlog item agile reports such as burndown charts, velocity charts, and
estimation
sprint reports are created by summarizing what happened using
historical project data and presented to the users in an intuitive
Fig. 2. The architecture of an AI-powered agile project management assistant and easily interpretable manner. Knowing what happened (e.g.
team velocity from sprint to sprint) is useful, but diagnos-
ing why something happened (e.g. why the team’s velocity
Traditional NLP techniques (e.g. Bag of Words) produce dropped significantly in some sprints) is even more useful.
very high dimensional and sparse vector representations. By AI equipped with machine learning can augment descriptive
contrast, latest advances in deep learning-based NLP tech- analytics by discovering patterns, identifying anomalies and
niques [4] such as word2vec, paragraph2vec, Long Short-Term detecting “unusual” events.
Memory (used in Google Translate), or Convolutional Neural 2) Predictive analytics: Most existing agile tools are not
Networks (used in Facebook’s DeepText engine) are able to yet capable of providing this advanced level of analytics. Two
generate dense vector representations that produce superior challenging areas are effort estimation and risk prediction that
results on various NLP tasks. Source code is another important are specifically for agile contexts. Machine learning techniques
source of project data. The Code Modeling component is are suited to build prediction models. For example, recent
responsible for learning vector representations which reflect work [6] used deep learning to estimate the size of user stories
the semantic and syntactic structure of source code. State- through learning a team’s previous estimates. Estimation tools
of-the-art statistical language modeling techniques, including could be used as a decision support system and takes part
deep learning models, have demonstrated their effectiveness in the existing estimation process (e.g. planning poker) or in
for source code and thus can be leveraged here [5]. a completely automated manner. Forecasting future risks re-
We have leveraged the powerful deep learning architecture, quires the capability of processing large amounts of historical
Long Short-Term Memory (LSTM) to automatically learn project data, memorizing a long history of past experience,
vector representations for both backlog items [6] and source and inferring the current “health” state of the project. Recent
code [7]. LSTM enables us to learn the semantics and syntactic work has moved forwards in this direction to predict delay
structures, particularly the long-term dependencies, existing in risks [9] or sprint delivery risks [8].
both natural text and source code. We will extend these models 3) Prescriptive analytics: This is the most advanced level
to learn representations for other textual artifacts such as in the project analytics stack. Using the results from descrip-
product visions, sprint goals, and developer communications. tive analytics and predictive analytics, prescriptive analytics
Any useful AI machinery must take into account the capa- recommends the best course of actions for agile teams in a
bility and dynamics of agile teams. Obtaining a representation specific situation. We identify here three important areas in
for a team requires modeling of its members (e.g. developers). agile PM that prescriptive analytics would be useful:
A developer can be represented through the project artifacts • Backlog item identification: Using the NLP component
they have involved with, such as the backlog items they have in the representation learning engine, prescriptive analyt-
completed or the code they have written. The Feature ics will automatically process and extract new backlog
Extraction and Aggregation extracts all the vector items from different data sources such as a requirement
representations of the artifacts related to a developer, and specification, new feature requests from customers, bugs
learn to aggregate them to form a vector representation of reported by end users, previous bug fixes, discussions
the developer. It also aggregates vector representations of among agile teams (e.g. technical debts, design changes
different artifacts (in both unstructured and structured forms) or action items from retrospective meetings), end users’
into a unified representation. A number of feature aggregation reviews of the product, and even experiences from pre-
techniques proposed in recent work [8] which derive vector vious projects. It will also able to recommend inter-
dependencies between new item and existing ones using for hyper-parameters tuning of machine learning models used
machine learning and representation learning. in the analytics engine. Search-based software engineering
• Backlog item refinement: prescriptive analytics will sug- techniques can be leveraged to build the optimization engine.
gest how a user story is split into a smaller user stories
D. Conversational dialog engine
or how a user story is decomposed into tasks. Learning
decompositions is highly challenging since it requires a The conversation dialog engine is envisioned to converse
background knowledge. It is still a challenging topic in meaningfully with agile teams. It is a form of a software
AI and machine learning. chatbot [13], acting as an interface between the users and the
• Risk mitigation: Using results from predictive analytics, remaining part of the AI system. The chatbot can be asked
prescriptive analytics recommends a course of actions to different types of questions, such as “Show me your estimate
take advantage of a future opportunity or mitigate a future of this user story” or “Can you help split this user story?”.
risk and shows the implication of each decision option. Through conversations with the users, it receives input and
requests, and passes them to relevant engines in the system.
C. Reasoning capability Future chatbots can also be trained end-to-end [14].
Reasoning is the capacity to infer new knowledge by IV. N EXT S TEPS
algebraically manipulating existing knowledge base to respond We are developing prototype tools to realize each compo-
to a query [10]. Traditionally, it works on symbolic knowledge nent of the proposed AI-powered agile project management
representation through the means of induction or deduction. assistant. We plan to first evaluate it using our existing
This permits domain knowledge provided by agile teams (e.g. dataset of 150 open source projects. We will also collaborate
project rules). Deep neural reasoning offers an alternative for with our existing industry partners to perform an evaluation
producing answers from sub-symbolic (vector) representation on commercial software agile projects. We however believe
[11], which are output of the representation engine. Inferred that AI will assist, not substitute, human teams. Individuals,
knowledge, after being validated, can be put back to enrich the interactions, and collaboration are still the key elements of
knowledge base. The reasoning capability of our AI system is project success. AI can serve as a distinctive accelerator for
provided by two engines: planning and optimization. agile teams and thus help increase project success rates.
1) Planning engine: Planning for a sprint can be formulated
as an AI planning problem in which the initial state is the R EFERENCES
state of the project and the product prior a sprint, a goal [1] R. Hoda, N. Salleh, and J. Grundy, “The rise and evolution of agile
state is specified in the sprint’s goal, and selecting items from software development,” IEEE Software, 2018.
[2] CollabNet and VersionOne, “The 12th annual state of agile report,”
the product backlog can be viewed as the act of choosing Tech. Rep., 2018, https://explore.versionone.com/state-of-agile.
plan operators to be executed from the initial state to a [3] M. Cohn, Agile estimating and planning. Pearson Education, 2005.
goal state. The planning engine needs to consider a range of [4] C. D. Manning, “Computational linguistics and deep learning,” Compu-
tational Linguistics, 2016.
input such as the existing product backlog items, the sprint’s [5] M. Allamanis, E. T. Barr, P. Devanbu, and C. Sutton, “A survey
goal, the existing codebase, the team’s capacity and previous of machine learning for big code and naturalness,” ACM Computing
performance in previous sprints. These data are often not Surveys (CSUR), vol. 51, no. 4, p. 81, 2018.
[6] M. Choetkiertikul, H. K. Dam, T. Tran, T. T. M. Pham, A. Ghose,
formally expressed. The representation learning engine learns and T. Menzies, “A deep learning model for estimating story points,”
to generate these data into vector representations, but further IEEE Transactions on Software Engineering, 2018. [Online]. Available:
formal encoding would also be needed. DOI:10.1109/TSE.2018.2792473
[7] H. K. Dam, T. Tran, T. Pham, S. W. Ng, J. Grundy, and
In addition, the plan needs to be executed in a manner that A. Ghose, “Automatic feature learning for predicting vulnerable
is robust and resilient to changes. The challenge is not only to software components,” IEEE Transactions on Software Engineering,
be flexible enough to deal with immediate impediments to the 2019. [Online]. Available: DOI:10.1109/TSE.2018.2881961.
[8] M. Choetkiertikul, H. K. Dam, T. Tran, A. Ghose, and J. Grundy,
sprint execution, but to also anticipate future states of affairs “Predicting delivery capability in iterative software development,” IEEE
that might impede sprint execution or the achievement of the Transactions on Software Engineering, vol. 44, no. 6, pp. 551–573, June
sprint goal. Impediments to the successful sprint execution 2018.
[9] M. Choetkiertikul, H. K. Dam, T. Tran, and A. Ghose, “Predicting the
can appear in many forms. For instance, a task might not delay of issues with due dates in software projects,” Empirical Softw.
be completed by the due date, preventing other dependent Engg., vol. 22, no. 3, pp. 1223–1263, Jun. 2017.
tasks from being started. Hence, the relationship between a [10] L. Bottou, “From machine learning to machine reasoning,” Machine
Learning, vol. 94, no. 2, pp. 133–149, 2014.
sprint plan and its operating environment can been seen as [11] H. Jaeger, “Artificial intelligence: Deep neural reasoning,” Nature, vol.
adversarial. Recent successful work in deep reinforcement 538, no. 7626, p. 467, 2016.
learning (e.g. [12]) can be thus leveraged to build this part [12] V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G.
Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski
of the planning engine. et al., “Human-level control through deep reinforcement learning,”
2) Optimization engine: The optimization engine helps the Nature, vol. 518, no. 7540, p. 529, 2015.
planning engine to compute the optimal set of actions given a [13] C. Lebeuf, M. Storey, and A. Zagalsky, “Software bots,” IEEE Software,
vol. 35, no. 1, pp. 18–23, January/February 2018.
certain situation. For example, it can be used to compute the [14] I. V. Serban, A. Sordoni, Y. Bengio, A. C. Courville, and J. Pineau,
optimal selection of backlog items for the upcoming sprint “Building end-to-end dialogue systems using generative hierarchical
given multiple constraints and objectives. It can also be used neural network models.” in AAAI, vol. 16, 2016, pp. 3776–3784.

View publication stats