Director Kevin Desouza employs sentiment analysis for tracking the progress of large-scale public sector projects from public documents
Kevin Desouza has co-authored a paper with Sandeep Purao and Jonathan Becker, both based at the College of Information Science and Technology at Penn State University, which analyzes the using sentiment analysis. The paper titled “Investigating Failures In Large-Scale Public Sector Projects With Sentiment Analysis” will appear in a special issue of e-Service Journal.
The research aims to determine early indicators of when large-scale public sector projects begin to go off the rails. Using historical analysis of 
the IRS Business Systems Modernization, a project that has already spanned a decade and consumed more than $3 billion, the paper explores quantifying stakeholder Sentiments and Confidence from documents, with a view to exploring how such measures may offer early indications of project progress and assist managers to prevent undesirable future outcomes.
The research views the large body of stakeholders working on large-scale public projects as possible human sensors, who display a nuanced perspective of the project. The paper posits that it is possible to track changes in stakeholder perspectives by examining publicly available documents and following the changes in perspectives as early indicators of project progress.
This research project is illustrative of the policy informatics work being done at the Metropolitan Institute. Information on public sector projects is available in abundance, yet this information is seldomly analyzed to proactively improve the performance of the projects. This research takes the first step in showing how mining of public information can lead to clues about the health of a public sector project. The methodology can be described as archaeological research that utilizes publicly-available stakeholder-generated documents as key sources of information. Each provides expressions of attitudes and sentiments by different stakeholders through the project lifecycle. The researchers analyzed almost 350 documents for a set of specific tags that reflected a positive, negative, understated, or overstated sentiment.
This technique can be used in conjunction with other assessment tools and is particularly useful in projects without access to primary data for evaluation. The paper finds that there are nuances of confidence and sentiment measurable across the life of the project that could have helped better understand how the different stakeholders perceive the project. The paper finds sentiment deteriorating every three years, roughly corresponding to changes in leadership on the project. The findings also uncover a possible correlation between sentiment improvement and greater stakeholder engagement. The paper maps the general confidence (often quite low) versus the sentiment (often quite high), and explores what this may tell public sector planners about managing large projects. Armed with this information, a governing body may be able to gauge project progress using measures other than the traditional milestones and meeting requirements or could use trends in this information to guide decisions and re-evaluations during critical points in the project timeline.
As Desouza notes, “the next step is to find ways to incorporate this technique for real-time project tracking. Can we begin to analyzed these reports as they are released, and then find indicators of project management challenges that are on the horizon. If you think about it, most large-scale public sector projects consume billions of dollars and have national, or state-level, impacts. Anything we can do to improve the success of these projects will help save millions in taxpayer dollars.”