Why a Systems Assessment Can Make or Break Your ERP #JoelKallmanDay

Introduction

A systems assessment exposes what changes need to be made to get the most out of your new ERP. It highlights what existing systems can now be replaced or retired and which need to remain to fill gaps the ERP cannot address.

This effort is often underestimated. Consider the following:

A 2025 survey by MuleSoft revealed that companies use an average of over 900 applications across the enterprise.

This figure includes homegrown, commercial-off-the-shelf (COTS), Mainframe, .NET, On-Prem, Cloud and SaaS applications and interfaces. Although the survey only included large companies with over 1000 employees, it’s still a staggering number.

But it’s not far-fetched. Here’s a real world example.

A recent client with over 600 systems in place, was comparing SAP versus Oracle Fusion ERP. As part of the evaluation, we analyzed how each ERP would impact the system landscape. The ecosystem was full of duplicity. For example, the company had eight distinct, siloed ERPs, one for each line of business. Each ERP performed similar functions, resulting in an eight-fold increase in integrations, licensing costs, and maintenance.

Additionally, 20% of systems were homegrown or on a Mainframe stack. Other systems, acquired through past mergers, were not well understood or considered too fragile to move or automate, impeding the company’s ability to scale and support new capabilities. The application landscape became too large to manage efficiently.

I highlight my client’s experience to show that a systems assessment not only determines what systems are impacted by the new ERP, but also exposes the overall health of your ecosystem. In total, it can deliver:

1. ERP Compatibility Assessment - identifies redundant systems or those with overlapping functions to the new ERP. If a system cannot be replaced by the ERP, can it integrate with it, and if so, what are the master data and transaction touch-points?

2. Risk Assessment - evaluates the health of a system, e.g., is it up-to-date, regulation-compliant, secure and supported?

3. Sustainability Assessment - evaluates the system’s test of time, e.g., is it integration-ready, configurable, maintainable, performant, scalable, affordable?

A systems assessment will produce an actionable Decision Matrix of what systems needs to be integrated, retired or replaced.

3 Key Takeaways ✍️

In case you are pressed for time, here are three key points I hope you walk away with:

These tools will miss:

1. Work performed in Spreadsheets - all downloaded and manually-tracked spreadsheets used to perform work.

2. Work performed by Human APIs - manual data entry between systems. Not all systems are integrated. Per the MuleSoft survey above, only 28% of systems are integrated, across large enterprises.

There are too many pitfalls when tracking entries and changes across hundreds of columns and rows. Which one of us has not experienced:

• “Who has the latest version?”; “Can you please check it back in?”

• File corruption or loss of data.

• Duplicate versions after someone forgets to check in their edits.

• Not everyone is familiar with VLOOKUPs and can accidentally break them

• Typos, leading to inaccurate conclusions. Did you know that manually-typed data has a 2-5% error rate?

Instead, Use Oracle APEX or another low code tool to track and maintain this crucial artifact. 👉👉 NOTE: this post is long so I wrote another post illustrating how I used Oracle APEX to create a systems assessment app here.

Yes, it’s hard to start, hard to complete and time-intensive. It requires access to people who understand the dependencies and complexities of each system and demands meticulous documentation.

Many companies get stuck in analysis paralysis, while others don’t know where or how to start. Perfection is the enemy here - even an 80% completion rate is a huge win given the limited information available to most teams.

Start with a small subset of systems and a few parameters (see the Criteria section below for a sample list).

How We End Up With So Many Systems

How do companies end up with hundreds of systems? Here’s a list of all the reasons I’ve collected over the years:

1. Mergers and acquisitions play a big part, where systems from newly-absorbed companies are not integrated into the existing ecosystem.

2. New systems are introduced because existing ones cannot accommodate the needs of a new line of business

3. “We’re stuck on an old version because upgrading customizations is costly and disruptive.”

4. Technical debt. Typically, when a company transitions to a new system, the old legacy system is kept running.

   👉 Here’s why:

   1. Other legacy systems rely on the marked-for-retirement system. Either the legacy systems do not have the capability to integrate with new systems or it's not worth the cost or risk to integrate, so the marked-for-retirement system remains.

   2. Mainframe and custom code apps. Often, no one left knows how these systems work. The adage, “if it’s not broken, do not touch it”, applies here.

   3. And the most frequent cited reason: "We've relied on it for so long, what if we need it for something? Let's just wait a bit longer and we'll retire it in a few months".

      Those few months turn into years. A new team comes in and has no idea what that system was ever used for, but again, afraid to touch it, so it remains in perpetuity.

What You Can Expect From An Assessment

An assessment exercise is one of those where you will get out what you put in. When complete, your assessment can deliver:

1. a complete, up-to-date view of the company’s current tech assets

2. An unbiased assessment (a Decision Matrix) of what apps need to be upgraded, integrated, replaced, consolidated or retired

   1. A listing of what can and should be replaced by the ERP

   2. A listing of what needs to integrate with the ERP

   3. A listing of systems that pose the highest risk to your ecosystem

3. With this Decision Matrix in place, you can:

   1. design a multi-phase roadmap, outlining which systems to tackle first

   2. estimate total implementation, integration, decommissioning, licensing and maintenance costs

   3. estimate total cost savings due to a reduced footprint (lower licensing and capital and operating expenses due to less legacy upkeep)

4. Finally, the executive team will be interested in aggregated summaries, analyses and recommendations including:

   1. A risk assessment analysis - systems that pose the highest risk to your ecosystem, with mitigation recommendations

   2. A technical debt analysis - a listing of technical debt that poses the highest impact to your program, with recommendations on how to mitigate and manage.

   3. Summary of Findings - key observations from engineer and support team interviews, such as:

      1. “We have a dependency on key resources for technology or architecture skills:

      2. “We lack process and standards relating to patch levels”

      3. “We lack password and access management - some admin passwords are still ‘password’, shared with many employees and contractors and never been changed.”

      4. “Backups and DR never attempted or successfully tested for over 40% systems”

      5. “We lack a robust Helpdesk ticketing system; have over 65% open and unresolved tickets.”

   4. Recommendations & Mitigations - represents mitigations to findings above, such as:

      1. “Establish Privileged Access Management. Enforce password updates every 90 days, enable multi-factor authentication for critical systems.”

      2. “Formalize a Disaster Recovery Program. Establish testing plan by system priority.”

👉 Armed with these insights, the next step is to understand how the ERP and system assessments connect - and why one should come before the other.

An Essential Prerequisite

Here’s why: you have to understand what you need and why - that’s the purpose of the ERP assessment. It maps your business processes and requirements against what the ERP can and cannot do.

With the ERP assessment in place, you can move on to the systems assessment to evaluate which systems can truly be retired versus which ones need to remain. Those that need to remain are kept because:

1. they fill gaps the ERP cannot address or,

2. another essential system depends on them

• To expand on point 2 - If the ERP assessment recommends replacing all eight existing legacy ERPs, the system assessment verifies whether this is feasible. For example, if one essential legacy system, outside the ERP project scope, depends on two of the eight ERPs, those ERPs cannot be retired until the dependency is resolved.

💰💰💰This is why system assessments pay off. It is best to identify dependencies ASAP so that integration, licensing and other costs are known in the beginning of the program.

How To Begin 🛠

1. Take inventory of all systems. Include not just applications but also spreadsheets used for business work, interfaces, APIs, and human-API activities.

2. Determine how you will assess each system. Consider criteria such as security, compliance, scalability, reliability and performance. I’ve provided a tabular list of the most common criteria below 👇

3. Apply weight factors. Weighting helps reflect relative importance. For example, if ‘HIPAA compliant” is three times more important as “IPv6 readiness”, a simple ranked list doesn't convey this importance, but a weighted list (e.g., HIPAA = 6, IPv6 = 2) does.

4. Rate each system. Assessments are a team effort. You can divide up the effort by:

   1. Expertise: security teams assess security criteria, operations teams assess reliability and performance, etc.

   2. Comprehensive Review: Every team member rates all systems, then discusses variances. This second approach takes longer but can bring about the most information and learning.

5. Analyze, summarize and present. With each system rated, your tool of choice can help you analyze, summarize and present the results. My tool of choice is Oracle APEX, as illustrated on the screenshots below. I’ve provided a screenshot of a spreadsheet for comparison purposes.

Spreadsheet version of the app above.

You can see I have categorized my 20 parameters by ERP compatibility, risk and sustainability considerations.

The filters above and the charts below auto-generated by the Oracle APEX tool when I uploaded my spreadsheet to the APEX tool:

For the purpose of this exercise, I used the following ratings for each assessment:

• ERP Map & Gap Assessment

  • Duplicate - system’s functionality can be replaced by the new ERP

  • Obsolete - functionality no longer relevant

  • Fills Gap - functionality the ERP does not cover

• Final Disposition Assessment

  • Retire - duplicate, high-risk, or obsolete; dependencies cleared.

  • Retain- fills a gap and poses low security risk

  • Upgrade - fills a gap but poses high security risk or low sustainability. Recommend to upgrade or replace.

Finally, Oracle APEX automatically provides me the ability to create new and edit my existing records as my assessment evolves:

Once your process is set, the next step is to agree on what you’ll measure — the assessment criteria.

Assessment Criteria

An important task as a team is to finalize the guiding principles to perform the assessment. Below is a listing of the most common parameters to evaluate systems. Not all are relevant to your enterprise so prioritize accordingly:

Once your criteria are finalized, you can begin scoring each system to reveal where modernization and/or retirement efforts will yield the biggest impact.

Conclusion

In summary, a global system assessment reveals the changes that need to be made in your environment to make the most out of the new ERP. It produces a Decision Matrix highlighting the systems that need to be upgraded, retired or replaced and applications that need to remain to fill gaps the ERP cannot address.

With this list of what systems need to stay or go, you can then estimate your integration, consolidation, decommissioning, licensing and maintenance costs for your ERP project and produce a multi-phase roadmap of which systems to tackle first.

The effort is significant but you only have to do the bulk entry of it once if you use a low-code tool like Oracle APEX to maintain a dynamic, continuously updated system inventory.

Every great ERP transformation starts with clarity. Your systems assessment is that first step. Good luck, and enjoy the discovery process! As always, reach out and let me know how it goes.