I attended the Bielik Summit 2025 today (unfortunately only virtually), and several speakers mentioned an MIT report regarding AI implementation in organizations. Naturally, I grabbed it (you can too: https://mlq.ai/media/quarterly_decks/v0.1_State_of_AI_in_Business_2025_Report.pdf).

The researchers examined over 300 implementations. The question is: were they the right ones? If half of them were attempts to apply AI to tasks requiring creativity, deep relationships, or extensive context, it’s no wonder they failed. The problem doesn’t lie in the technology. The problem lies in the choice of applications.

According to the report, 95% did not yield a return on investment (ROI) within the studied period: six months after deployment. Wait a minute – 6 months? That’s very little, even for classic IT investments.

Most IT projects start generating value after a year, not half a year. An ERP implementation? A year or two to stabilize. A new CRM? Minimum nine months for adoption. Process automation? Six months just for fine-tuning. And here we are measuring the ROI of generative AI after six months and declaring a 95% failure rate? This is not a reliable assessment. In my opinion, it’s too early for a verdict.

The report is focused on ROI and looks through the prism of whether companies that implemented GenAI tools started earning more. Okay, but that is still too harsh a criterion for such a short period.

Looking through this prism over only six months is, in my view, the same mistake as improper AI implementation itself.

The article highlights four patterns:

• Limited Disruption: Only 2 out of 8 major sectors show significant structural changes to adapt to AI.
• The Enterprise Paradox: Big firms lead in pilot volume but lag in scaling – similar to above, it’s harder for a large entity to change.
• Investment Bias: Budgets favor visible “top-line” functions (generating revenue) at the expense of high ROI in back-office support departments. This is also an error stemming from expecting more from the technology than it can currently offer.
• Implementation Advantage: External partnerships achieve double the success rate compared to internally built projects. Firms that implement AI daily have established procedures and know-how. Meanwhile, many companies simply ordered internal IT departments to start absorbing new technology without any training.

The report points out five myths and falsehoods about generative AI in enterprises:

Five Myths About GenAI in the Enterprise:

1. AI will replace most jobs in the next few years – Research showed limited layoffs.
2. Generative AI is transforming business – Adoption is high, but transformation is low.
3. Enterprises are slow in adopting new tech – Enterprises are very eager; 90% have seriously explored purchasing.
4. The biggest obstacle is model quality, law, data – In reality, the obstacle is that most AI tools do not learn and do not integrate with processes.
5. The best enterprises build their own tools – Internal projects (build) fail twice as often according to the report.

One of the implementation errors, according to the report, and a problem with technology adoption, is that AI does not adapt to the company; it lacks real Memory and the ability to learn.

As I wrote in a previous post [Link to your post], generative AI does not learn. The implementation process should clarify this limitation right at the start. Systems are built based on retrieving matched context for answers in real-time. We, who use these systems, and we, who create solutions utilizing them, have no influence on the models themselves or their inherent knowledge. It’s “out of the box,” and businesses must be aware of this.

Effective implementation occurs in steps. We release part of the functionality, test, deploy, add more, and repeat. A method of small steps, always with a Human in the Loop.

According to the report (and I agree completely), the determinants of successful implementations are: deployment flexibility, ongoing adaptation to needs, understanding workflow and processes in the organization where the system is deployed, limiting the initial impact on currently used tools, and clear boundaries for the data the system is to operate on.

AI has a limited context window, which means it can process a limited amount of text at once. Additionally, systems lose “attention” in certain content areas when processing several different tasks simultaneously; they will certainly forget something and hallucinate something else. Agentic action saves us from this a bit, but not fully.

To summarize: AI implementation in companies must be preceded not by widespread HYPE and marketing claims of “build an agent in five minutes,” but by analysis, testing, starting with simpler things, and scaling.