For many government clients, a failed COTS* software project can often feel like a crime scene. Clues that could have helped avoid a costly project failure are often revealed only after the fact, reminding all involved of the critical importance of careful due diligence in choosing a capable enterprise software provider.
To help avoid a failed project, put your best procurement detectives on the case to ensure your prospective vendor has not committed any software crimes (i.e. poorly executed, failed, cancelled or abandoned projects).
By simply examining the publicly available evidence and clues, your COTS Software Investigation efforts will ensure the success of your next enterprise system implementation, and prevent further software crimes!
*Commercial Off-The-Shelf
Click to view and enlarge (Ctrl+) full infographic
The absolute best place to start your investigation is by conducting a detailed examination of your prospective software provider’s track record.
Does the company have a proven and positive track record working successfully with jurisdictions of your size and with projects of comparable complexity? What have been the outcomes of their similarly-sized projects in the last two to three years, in particular?
You should also investigate whether there has been any history of failed or cancelled projects (including dates and with whom), what the perceived causes were and, even more importantly, if the vendor has any pending lawsuits or in-progress legal action? While some of these questions can be posed directly to your potential software provider(s), some of this information can, and should, be validated independently through your own web search.
The number one source of clues that a ‘software crime’ has been committed is the victims of such crimes, those unfortunate government clients that have incurred considerable time and expense to disentangle themselves from a failed project, or worse, those still operating within the crippling constraints of a failing project with under-performing software. Fortunately, for the savvy investigator, evidence of customer satisfaction and/or dissatisfaction is easy to find thanks to public customer review websites such as Capterra as well as publicly available coverage of failed government software projects that is easily accessible through a systemic and thorough online investigation.
In searching for these clues, ensure that you do not focus extensively on finding the ‘victims’ only however, and exclude those success stories that are clear evidence of happy customers well served by a proven provider. In this avenue of your investigation, focus on identifying additional sources of information beyond the obvious case studies and testimonials provided by the software vendor themselves. To gain a balanced perspective, providers should be asked whether they conduct customer satisfaction surveys, what type, how frequently, and if those survey results or testimonials are available for your review? According to the popular customer loyalty diagnostic, the Net Promoter Score, the single most important question in a Customer Satisfaction and Loyalty evaluation is: “On a scale of 1-10, how likely would you be to recommend this vendor to others?”
Additionally, search to see if the provider’s customers have ever won or received significant awards or industry recognition for their implementation of the vendor’s software?
Finally, be sure to use professional networking websites such as LinkedIn, or those facilitated by pertinent industry associations, to conduct outreach with your government counterparts, with direct or historic involvement with the prospective provider and their product line, to learn from their experiences. From product quality to project implementation approach, project management acumen to ongoing customer service & support; such conversations can reveal a diverse range of customer perspectives, so it’s important to dialogue with as many operational roles as possible, from the C-Suite to front-line users of the product solution.
Click to view and enlarge (Ctrl+) full infographic
With government cybersecurity threats escalating, all potential software solutions (and the prospective provider’s optimization of same) have to be evaluated through the lens of the solution’s capabilities to detect and defend to the rigorous compliance standards demanded of modern government operations.
For detailed best practices for conducting a thorough investigation of the cybersecurity capabilities and credentials of a prospective software provider, please refer to our Four Part Series on Cloud Security. For an immediate snapshot, refer to the following list of ‘must have’ cybersecurity requirements that you must be able to validate within your vendor’s proposed cybersecurity infrastructure:
Are all software crimes committed with malice? Not at all. In fact, many failed projects start with the best intentions of all parties involved. In such cases, it is often a case of systemic flaws within your prospective provider’s operational model that prevents the eventual solution implementation from achieving the entirety of your project requirements. For this reason, examining your prospective vendor’s operating model in detail is fertile ground for investigation, yielding clues in the areas of government experience, change management, project management, customer service, technical support, and more.
To start in the right direction, investigate whether the company employs any relevant industry Subject Matter Experts who can be leveraged into the Business Analysis or Change Management aspects of the project? It helps to have project staff who have obvious industry credibility and who are already up-to-speed on your business processes and needs. Do those in-house SMEs offer industry credentials and a broader, more enhanced perspective?
Further, is your envisioned provider a private or public company? There are pros and cons to working with each kind of company. Your research should help you determine if the provider’s project track record suggests that they value long-term customer satisfaction ahead of short-term profit margins and shareholder returns. Does the vendor have a history of exceeding established project budgets with unexpected and excessive change requests to maximize profitability?
Does the company have vertical specific experience working with comparable government agencies and projects, or are they more accurately described as ‘generalists’ who rely heavily on third party consultants and/or your internal staff to achieve fluency in your business rules and workflow optimizations? As this domain expertise extends to the product platform, does the vendor offer a diverse range of Saas, On-Premise, and Hybrid solutions, or are you more realistically looking at a ‘one size fits all’ offering that may lack the flexibility required to accommodate emerging technologies and business trends?
Click to view and enlarge (Ctrl+) full infographic
Properly planned and executed, a robust government enterprise software solution has the capability to deliver true generational value to staff and taxpayers alike. Achieving an extended product lifecycle of this nature requires a fiscally sound solution provider with the wherewithal to continually refine their technology and the customer service ethos to deliver exceptional support for a period of 25 years, 50 years, or more! For more information on this topic, please refer to our article: Evaluate the vendor’s product roadmap to ascertain longterm partner stability and project viability.
For specific clues to investigate to ascertain your provider’s financial health, the following indicators are an excellent starting point:
Enterprise software development is not the core area of expertise for most government agencies. For that reason, conducting a detailed investigation of a prospective software vendor’s approach to product development can seem a complex challenge. By investigating the following areas of discovery in conjunction with your internal technical staff however, a little investigative legwork can pay huge dividends in uncovering clues as to the product’s genuine suitability for your project requirements:
Annual Research and Development Investment
Profitable, innovative companies commit to an annual research and development investment in order to keep their company and products current, compelling and viable. What is the software provider’s annual commitment to R&D? What has been the annual investment in the past 2-3 years? What new products, innovations, or best practices did that R&D investment produce?
Hot Tip: Check the prospective provider’s financials to confirm the annual revenue percentage reinvested into ongoing product research & development.
Software Product Roadmap
What is the provider’s governance model for ongoing product development? Can the vendor provide a formalized, document Software Roadmap for your review? Going forward, can you anticipating having client input into the future direction of product development and feature upgrade prioritization?
Multiple Competing Product Lines
Some providers have adopted a high-growth strategy of acquiring other companies or smaller competitors as a means of expanding their client base. Ask the vendor how many different/similar products they are carrying and supporting? Ask if the company has formal Product Managers (request resumes – you may be surprised!) who oversee the planning, prioritization of product enhancements, forecasting and production of the software application? Supporting multiple software code streams, each with multiple versions in maintenance mode, requires separate streams of product management, development, QA and support resources. This can be expensive to maintain, cause resources to be distracted or spread thin, and contribute to a lack of focus on the core product. Ask if the provider’s exclusive focus is on their flagship product?
SOA Compliance / Software Interoperability
A published standard in your RFP against which software can be judged to be interoperable and “Services Oriented Architecture” (SOA) compatible will be of value to your organization. Define software requirements that will provide full integration with other third-party products/functions to enable the interchange of information and system compatibility across all in-scope business processes, thus providing your organization with full ownership of your data. Meeting this standard will also limit software integration and ongoing re-integration costs.
Low Code Platform
By reducing or outright eliminating the need for business users to write complicated code, low code based platforms have become the preferred software solution paradigm for government agencies looking to realize the following wins:
Is your provider’s product a legitimate low code platform? Investigate by asking the following: Can you bring in all the data you need? Can you define relationships between data being integrated and information from other systems? Can you filter or add rules on what data is displayed? Can you update systems of record based on user actions?
COTS or Configurable
Is your provider building your own custom solution bespoke to your exacting business requirements OR are you buying an existing COTS (Commercial Off The Shelf) solution that meets many, but not all, of your business requirements? Most importantly, have you considered the third option, COTS+?
A COTS+ Solution is the best of both worlds, providing the off-the-shelf feature maturity of a traditional COTS product with the underlying flexibility of configurable software. This powerful hybrid approach places COTS+ solutions in a procurement ‘sweet spot’ for government agencies seeking the product certainty and speed-to-market of an existing off-the-shelf solution in tandem with the extended lifecycle value of a low code development and workflow automation platform.
Click to view and enlarge (Ctrl+) full infographic
One of the key aspects that separates successful project implementations from the disappointments is the degree of compatibility that exists between client and vendor.
Does your organization and the provider share common ground in your respective use of recognized standards and/or professional memberships, certifications and affiliations?
Appropriate expertise indicators may include the following:
Maintaining a stable workforce means a provider can plan for and deliver projects more predictably and with greater efficiency without having to deal with significant project team churn or delay. The worldwide average for annual staff turnover in the software industry is in excess of 15 per cent, however some vendors experience higher rates of annual staff turnover.
On Glassdoor, former staff members of companies are free to comment on such aspects as company leadership, working conditions, salaries, company performance, and staff satisfaction. High-growth companies with a strong M&A roadmap may be over-extended or be struggling with cash flow or HR-related challenges. Merging company cultures creates staff stress and instability. Are there any present or recent mergers or acquisitions underway at the company?
A common thread in most software crime scenes is an underwhelming product implementation that fails to meet the project requirements anticipated upon live launch followed by a expensive support model leveraged to ‘fill in the feature gaps.’ Another common, albeit equally frustrating, scenario is the envisioned product solution passing muster initially but failing to achieve extended lifecycle longevity with support costs increasing while support quality decreases.
To properly investigate your prospective provider’s service & support model, look to the following areas for clues of a strong customer commitment:
Looking for more information? Subscribe to our CX blog below to stay in the loop on the latest news, best practices and emerging trends in government technology.
