Beyond The Hype: Harnessing Generative AI For Tangible Business Impact

Introduction

Since ChatGPT was launched on November 30, 2022, generative AI (GenAI) has been at the forefront of technological discourse. To many, it represents a disruptive force capable of redefining business processes and creating unprecedented efficiencies; however, some see it as an overhyped solution that has yet to live up to its promises, especially when early pilots fail to deliver the expected return on investment (ROI).

Many organizations have launched GenAI pilots, but a significant number of these initiatives have struggled to transition beyond the pilot phase, leaving decision-makers questioning GenAI's value. These reservations seem even more stark in light of initial high expectations. While assessments of GenAI's capabilities and limitations are becoming more realistic, organizations should still recognize its transformative potential when leveraged strategically.

There are noteworthy instances where existing GenAI technology has demonstrated its substantial impact. In specific, highimpact use cases, GenAI can deliver near-term benefits that are both scalable and broadly applicable. These successes suggest that while GenAI may not be a universal solution, it holds considerable potential to accelerate digital transformation when applied strategically.

This article will explore the challenges that organizations face in achieving ROI from GenAI investments and demonstrate how they can harness this powerful technology to drive near-term impact using two key applications:

1. AI-Powered Code Generation 
2. Parsing Unstructured Data Into Structured Formats

Challenges to Scaling GenAI Solutions

Early GenAI adopters and enthusiasts touted its ability to reason, automate complex tasks, generate creative content, and even revolutionize entire industries. This enthusiasm was fueled by seemingly rapid advancements in subsequent releases of large language models (LLMs) by OpenAI, Google, Anthropic and others, which demonstrated that machines could now generate text, code and even artwork with a level of sophistication previously unimaginable.

However, as organizations rushed to embrace this new technology, the reality of implementation began to diverge from the hype. While pilot projects often showcased impressive capabilities in controlled environments, scaling these solutions proved to be a different story. Issues such as data quality, model accuracy and hallucination, and challenges integrating with existing systems became apparent, hindering the transition from pilot to production.

Many GenAI pilots have failed because GenAI is not always the right solution for the problem at hand. While GenAI excels in content creation, human-like interaction and exploratory analysis, its effectiveness and cost-efficiency decrease in scenarios demanding precision, regulatory compliance, predictability and computational efficiency

Even in feasible applications, the following challenges may arise:

• Data Quality and Availability: GenAI models rely on high-quality and up-to-date data for effective fine-tuning and production use, but inaccuracies or incomplete documentation often lead to subpar results.
• Model Accuracy and Hallucination: Regardless of data quality, models may “hallucinate,” producing plausiblesounding but incorrect or nonsensical outputs, which can undermine trust and reliability. 
• Regulatory Compliance: Particularly in regulated industries (e.g., healthcare, financial services), ensuring that AI systems meet rapidly-evolving requirements adds complexity to deployment and limits the use of certain tools. 
• Compute Costs: Fine-tuning or training smaller models remains computationally intensive, and inference costs can become prohibitive in high-volume use cases (e.g., thousands of daily customer calls), especially for smaller organizations or those with limited IT budgets. 
• Enterprise Integration: Integrating GenAI solutions into existing IT infrastructure and workflows can be technically challenging and resource-intensive, and may introduce risks or disruptions to systems and processes. 
• Deployment and Operations: Protocols and systems for developing and managing GenAI solutions (LLMs and agents leveraging LLMs) are still immature, analogous to early-stage DevOps or AIOps tools, which increases complexity and risk for organizations deploying this technology.

Despite these challenges, GenAI has made a significant impact in certain sectors. Notably, tech companies have leveraged GenAI to optimize software development processes, resulting in substantial productivity and efficiency gains. Although an accurate number is hard to ascertain, many sources estimate that between 40 percent and up to 90 percent of all code at major tech companies is now generated by AI. In the following sections, we will explore high-impact use cases where GenAI has proven its value, shedding light on the opportunities it presents for driving digital transformation across sectors and functions.

High-Impact, Near-Term Use Cases to Drive Digital Transformation

While GenAI use cases like chatbots often capture the early focus of GenAI initiatives due to their visibility and customer-facing benefits, we believe that the real near-term value for a broader set of companies lies in applications such as code generation and the parsing of unstructured data into structured formats. These use cases not only present lower risks while offering immediate and substantial benefits, but also help overcome key challenges associated with scaling AI.

1. Code Generation

GenAI is particularly well-suited for code generation due to its proficiency in pattern recognition and language modeling, enabling it to automate repetitive coding tasks with high accuracy. This capability is supported by the vast datasets of code and programmer comments available on platforms like GitHub and Stack Overflow, as well as the relatively simple, structured syntax of programming languages compared to human language. However, while GenAI excels at generating boilerplate code and automating routine tasks, it may struggle with more complex, context-dependent coding challenges that require deep understanding of business logic or creative problem-solving.

The impact of GenAI in code generation is exemplified by Amazon Web Services, which utilized the technology to save over $260 million, equivalent to 4,500 developer years of labor. This not only drives incredible efficiencies, but accelerates the product development lifecycle and addresses key scaling challenges, enabling product managers and developers to focus on more strategic and innovative work.

2. Parsing of Unstructured Data Into Structured Formats

Unstructured data — such as text within emails, comments in systems of record, or free-form notes in medical records — represents a significant portion of the data that organizations generate and collect. Historically, transforming this data into a structured, consistent format has been a labor-intensive process that could take months, creating a significant bottleneck for broader technology and digital transformation efforts.

However, GenAI excels at understanding natural language, extracting key concepts and converting that content into structured formats, such as .csv or .json files, for integration into systems for storage, processing or analysis. As a result, it can streamline and automate this otherwise labor-intensive processes, reducing timelines from months to weeks or even days and addressing one of the most critical challenges in scaling digital and AI systems: the lack of a quality data foundation. By turning unstructured data into a consistent and usable format, GenAI helps create a more robust data infrastructure, enabling organizations to effectively deploy automation, digital and AI solutions at scale.

Recent example from a large healthcare services company

• The company faced significant challenges while migrating from legacy systems to a leading CRM platform. Critical provider information — such as availability, capabilities and services — was inconsistently recorded in comment fields in the old system, causing delays and inefficiencies, and impacting patient service. The company's IT team struggled for months to convert this data using traditional coding techniques, but the variability led to poor accuracy. By leveraging their existing ETL tools and a secure API endpoint to an LLM, we were able to resolve this issue, completing the data migration in just a few weeks.

Industry- and Function-Specific Applications and Benefits

The application of these two, broad use cases for GenAI spans industries and functions. The following list of examples is not exhaustive but is intended to provide a starting point to consider where these approaches could add value in your organization.

Industry Examples

1. Healthcare: Transform unstructured data from electronic health records systems like Epic into structured formats, enabling more efficient analysis of patient histories and improving diagnostic accuracy.
2. Travel Industry: Refactor large codebase for legacy reservation systems to reduce processing times and improve integration with modern APIs, enhancing system scalability during peak travel seasons.
3. Legal Industry: Transform unstructured data from case notes and depositions into structured formats, enabling more efficient extraction of key data and metadata for use in document management systems.
4. Insurance: Transform data from claims reports and adjuster notes into structured formats, enabling more efficient extraction and analysis of key information for use in claims management systems. 
5. Education: Convert unstructured data from student evaluations, research notes and lesson plans into structured formats, enabling educators to more effectively track student progress, optimize curriculum development and improve educational outcomes.

Function-Specific Examples

1. Technology: Refactor and translate legacy systems written in outdated programming languages like COBOL, FORTRAN and Pascal into modern, maintainable code that current-day developers can work with.
2. Technology: Reduce time to deliver bug fixes and new features for technology across the business by automating scripting using AI tools, saving time, improving service delivery to business units and enhancing system security. 
3. Finance: Parse receipts and expense reports submitted through systems like Concur, automatically categorizing expenses and flagging anomalies for review, thereby streamlining the expense management process. 
4. Business Intelligence and Reporting: Business analysts can rapidly develop more sophisticated analyses and custom reports by leveraging programming languages like Python based on analysts' deep knowledge of business processes and data structures. 
5. Human Resources: Accurately parse and analyze data from resumes, cover letters and candidate emails, extracting contact information as well as skills and experiences in a consistent way for use in applicant tracking systems. 
6. Supply Chain: Transform unstructured data from bills of materials or bills of lading to enhance inventory tracking, to improve supplier coordination and streamline logistics planning. 
7. Digital Product Teams: Rapidly generate code to test new website or app features, accelerating new product development and innovation.
8. Cybersecurity: GenAI can analyze unstructured log data from cybersecurity tools like Splunk, enabling identification of patterns indicative of potential threats, and automating responses using machine learning or other tools to mitigate risks.

Strategic Considerations for Implementation

To fully leverage GenAI for accelerating digital transformation, organizations need a strategic approach tailored to the specific GenAI capability and aligned with your business objectives, governance approach and technology requirements. Here's how to prepare and implement these technologies effectively.

1. Getting Started With GenAI for Code Generation

• Select a Tool and Establish Secure Access: Each GenAI tool offers different strengths in code generation. For example, GitHub Copilot is highly integrated with development environments and excels at generating boilerplate code, while tools like ChatGPT or Anthropic's Claude offer broader conversational capabilities that might assist in more diverse tasks. Access to these tools often comes with subscription or usage costs, which should be factored into your budgeting and resource allocation. 
• Establish Guardrails and Policies: Define what types of code and data can be processed by GenAI tools, set access controls and monitor usage to prevent unintended consequences, such as code vulnerabilities or intellectual property issues. This proactive management is key to maintaining security and compliance, particularly in sensitive or regulated environments.
• Pilot and Measure: Design small-scale pilots focused on specific code generation tasks to test the effectiveness of GenAI tools in your environment. Measure outcomes such as time saved, error reduction and overall productivity gains. These pilots will provide valuable insights into how GenAI can best be utilized within your organization and inform broader implementation strategies.

With a better understanding of how GenAI can augment your development capacity, you can begin to take a more comprehensive and strategic approach:

Accelerate or Reprioritize Projects: Based on initial lessons learned, identify opportunities to accelerate existing projects or reprioritize initiatives. You may also reconsider reliance on certain vendors or external providers if GenAI enables more efficient in-house development. This may have significant implications on your strategic approach to IT, procurement and more.

2. Accelerating Data Transformation With GenAI

• Identify and Prioritize Unstructured Data Sources: Begin by identifying unstructured data within systems like CRM, EHR or financial databases. Prioritize these sources based on business needs, such as improving customer insights or enhancing compliance, value and complexity.
• Consider Data Security and Compliance: For sensitive data, use GenAI models that meet regulatory requirements. Healthcare organizations, for example, must ensure that any GenAI tools used with protected health information are HIPAA compliant, potentially requiring proprietary models or alternative approaches, which can substantially impact cost and complexity.
• Design Targeted Pilots: Develop pilots to test GenAI's ability to extract key data from unstructured sources. Include rigorous testing to ensure data accuracy and security, especially when dealing with sensitive data. Consider the complexity of the data and the required level of accuracy for downstream processes. Prompt engineering is critical to accurately parsing unstructured data, and you will likely need to experiment with multiple iterations. Given the cost to leverage these tools (inference), you will want to test on a limited number of records before expanding to thousands or even millions of records.

3. Improving Data Quality and Accessibility

While GenAI can structure unstructured data, addressing the underlying causes of data quality and accessibility is critical for long-term success and scaled digital transformation.

• Process and User Interface Improvement: If unstructured data arises from inconsistent data entry (e.g., sales reps entering client information into comment fields), consider redesigning the data entry processes, providing training and modifying user interfaces — for example, by creating additional fields with drop-downs rather than free text — to capture data consistently at the source. This proactive approach will reduce future governance issues and lower costs over time.
• Data Ingestion and Pipelines: Ensuring data integrity requires robust data ingestion and pipeline practices. Automated pipelines with validation checks, schema enforcement and error-handling mechanisms help maintain clean, structured data. Incorporating metadata management to track data lineage and quality further enhances transparency, enabling quicker issue resolution and supporting accurate, secure and reliable analytics.

The Future of GenAI

The landscape of generative AI is rapidly evolving, with significant advancements poised to further enhance its capabilities and value. Tools supporting the development lifecycle of GenAI, as well as emerging “agentic architectures,” are enabling safer and more effective use of large language models in automating complex, end-to-end processes. Additionally, GenAI is increasingly being embedded into existing applications across various enterprise systems, making it more accessible and integrated into everyday business operations.

As these technologies continue to advance, the applicability and value of GenAI are likely to expand, offering new opportunities for innovation and efficiency. Organizations should adopt a continuous learning approach to ensure they stay ahead of these developments and maximize the benefits of GenAI. By regularly reassessing their use of GenAI and staying informed about the latest tools and best practices, businesses can maintain a competitive edge in their digital transformation efforts.

While GenAI has already demonstrated significant business impact in certain areas, its full impact is yet to be realized. Organizations that proactively adapt to the evolving capabilities of GenAI and integrate them strategically into their operations will be well-positioned to drive innovation and achieve sustainable growth.

How A&M Can Help

At A&M, we partner with clients to solve complex business challenges by applying AI where it can deliver the greatest impact. We not only help organizations deploy AI solutions like GenAI effectively but also guide them in building the necessary infrastructure, governance and skills to safely harness the potential of emerging technologies. By aligning technology investments with strategic business goals, we ensure that our clients are prepared to navigate the evolving AI landscape with confidence and success.

i. Goldman Sachs, “Gen AI: Too Much Spend, Too Little Benefit?,” Goldman Sachs: Top of Mind, no. 129 (June 2024): 2-24, https://www.goldmansachs.com/images/migrated/insights/pages/gs-research/gen-ai--too-much-spend,-too-littlebenefit-/TOM_AI%202.0_ForRedaction.pdf.

ii. Alvarez & Marsal, “Navigating the Hype: The Strategic Use of Generative AI in Business,” Alvarez & Marsal Digital & Technology Services (June 2024), https://www.alvarezandmarsal.com/sites/default/files/2024-06/445063_CPI-DTS_DTS_Strategic%20Use%20of%20Generative%20AI%20TL_03.pdf.

iii. Thomas Dohmke, “Github Copilot is generally available to all developers,” Github Blog (May 2024), https://github.blog/ news-insights/product-news/github-copilot-is-generally-available-to-all-developers/#:~:text=In%20files%20where%20it%27s%20enabled,we%20expect%20that%20to%20increase. 

iv. Paul Graham, on the percentage of code now written by AI according to his queries; “The number is rarely lower than 40%. For some young programmers it's 90%.” Reddit, accessed Sept. 13, 2024, https://reddit.com/r/singularity/comments/1eq77s/what_percent_of_code_is_now_written_bt_ai_i_ask/.

v. Andy Jassy, “One of the most tedious (but critical tasks) for software teams is updating foundational software,” LinkedIn (August 2024), accessed Sept. 11, 2024, https://www.linkedin.com/posts/andy-jassy-8b1615_one-of-the-most-tediousbut-critical-tasks-activity-7232374162185461760-AdSz/?utm_source%3Dshare%26utm_medium%3Dmember_ios. 

Originally published 24 September 2024

The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.