1. Why Testing is increasingly important
Software Quality Assurance is an essential aspect in todays ever accelerating business process world. Order and delivery processes are integrated on vendors and customers IT sites and are more and more just in time to decrease storage cost and improve production speed. Major IT failures add up to millions and billions when business processes get interrupted. As a conclusion everybody needs to test to insure stability of the most important business processes. According to Cap Gemini’s world quality report (2019/20) the spending on software development process steadily grows whereas spending on human resources slowly increase and 23 – 28% of the development budget is allocated in testing.
On the other hand still most companies using ERP software in their daily business perform the necessary testing manually. Testing and tasks may be supported by Excel sheets with test data and instructions but are mostly done with high redundancy or meeting only 10-30% of the relevant, most-at-risk business processes.
The Testing Dilemma
Manual tests are very costly and time intensive and bind human resources to non-business tasks – losing pace in the worldwide competition.
Modern test automation tools can help significantly to improve this process and make testing much more efficient and effective at the same time.
2. A new Partnership in Test Automation
In autumn 2020 SAP decided to resell Tricentis testing software as SAP Solution Extensions as SAP Application Testing Solutions by Tricentis. The Tricentis solution portfolio has proven state-of-the-art modern test automation with high focus on efficiency and effectiveness in the testing process. In the last years Tricentis has always shown best positions in Gartners and Forresters magic quadrant and at the same time continuously grown their customer base worldwide.
SAPs top reasons to choose Tricentis as its Official testing solution are depicted in the following picture:
Top reasons for Tricentis
Many customers of the top10 market leaders in business branches like Banking, Financial Services, Energy/Utilities, Insurance, Telco’s and Global Services rely on Tricentis testing solutions and customer base has grown over 2000.
Tricentis products are part of the SAP pricelist and the P&T product portfolio (with different or adjusted product names).
3. The Tricentis Product Portfolio
The Tricentis product portfolio is mainly consisting of three solutions:
SAP Change Impact Analysis by Tricentis (CIA) – (product name at Tricentis: LiveCompare)
SAP Enterprise Continuous Testing by Tricentis (ECT) – (product name at Tricentis: Tosca)
SAP Enterprise Performance Testing by Tricentis (EPT) – (product name at Tricentis: NeoLoad)
SAP application testing solutions from Tricentis are available in the cloud to complement the RISE with SAP packages. These solutions pass rigorous testing process (premium qualification) and offer long term return on investment. SAP Solution Extensions are built into the SAP roadmap to ensure customer investment is future proof. Customers get SAP Commitment — SAP Solution Extensions are supported by the SAP support program, which provides one consistent support delivery experience, one set of comprehensive lifecycle management tools, and one comprehensive infrastructure for mission-critical support.
In a nutshell, SAP application testing solution by Tricentis addresses 3 key areas as part of the test management.
SAP Change Impact Analysis by Tricentis covers the aspect of selecting the software parts that have to be tested on a special upgrade/update or hotfix to the SUT (System under test) and answers the question: “What to test?”
SAP Enterprise Continuous Testing by Tricentis covers the aspect of functional testing: “Does it work?”. This is the original flagship solution extension from Tricentis that was developed on the black-box testing approach with model based test automation to achieve better stability and flexibility on software changes and test case maintenance.
SAP Enterprise Performance Testing by Tricentis covers the aspect of performance testing and puts the SUT under load (load test). It can emulate hundreds and thousands of users accessing the same service at the same time, answering the question: “Does it scale?”.
All three products are well integrated and firmly fit together in the continuous development process of modern software design:
The software design loop
P.S. SAP has added Enterprise Data Integrity Testing (EDIT) to the Tricentis product portfolio in 2021. For a description please look here: https://blogs.sap.com/2022/12/07/sap-enterprise-data-integrity-testing-by-tricentis-edit-overview/
4. SAP Change Impact Analysis by Tricentis (CIA)
Change Impact Analysis is an external tool acquired by Tricentis in 2019 that will analyze Software Updates/Upgrades or Supportpackages and Hotfixes with respect of most-at-risk processes thus answering the question: “What to test”. In this form it delivers important information to the risk-based testcase design and improves overall efficiency and effectiveness in the testing process. It’s a modern application based on AI and will compare software code side-by-side to derive what parts have changed and have to be tested before upgrading a SUT in production. It integrates well with ECT while providing a fit/gap analysis of test cases and test data to reveal where there are gaps in the test case repository to cover all processes at risk.
- CIA in many cases can reduce the testing scope and effort significantly with smart impact analysis that identifies existing test coverage and gaps.
- Increase speed to market by identifying all the differences that impact business processes, system integrations, custom code, security and governance.
- Release with limited defects in production by confidently knowing which custom changes will impact production.
With CIA you will be able to concentrate on the most-at-risk business processes and reducing redundancy in test cases significantly. Especially in repeating test cycles during development waves you want to focus on the change and the most important part to be fast and agile – CIA is your tool of choice: (data provided by Tricentis from customer projects)
The so-called “treemap” analysis of CIA will show you at first glance which processes and modules are impacted by any change and which of them are most-at-risk, so you can focus on these parts to perform a very efficient testing:
The blue squares depict the processes that are changed and are most-at-risk. The areas in dark grey are also affected by change but are not as important to your daily business, so they deserve less priority in testing. The areas in light grey are not affected at all and don’t need any testing.
As CIA is integrated into ECT you will get a so-called fit-gap analysis of your testcase repository to show which tests already contribute to the testing task in scope and where there are gaps in the testcase repository which have to be filled to cover all processes most-at-risk (blue areas):
The necessary testcases are automatically identified and put into the requirements section in ECT’s Tosca-Commander, the cockpit of ECT for test automation.
Furthermore you can analyze all aspects of the CIA-findings in deep detail, from application area, transaction level, usage frequency, level depth in call-structure and even down to codeline comparison:
5. SAP Enterprise Continous Testing by Tricentis (ECT)
SAP Enterprise Continous Testing by Tricentis is the solution for functional testing. This is the flagship product that was developed in 2007 by the founder of Tricentis, Wolfgang Platz around the so-called black-box testing approach with model based test automation. In comparison to the older script based approach the model based test automation has many advantages in maintenance effort for test cases and flexibility.
SAP Enterprise Continous Testing by Tricentis covers several aspects of functional testing and can be seen as a modern test automation suite in continuous development and software delivery process.
ECT - whats included
Model based test automation
The model based test automation approach abstracts from the implementation of the SUT by introducing a (simplified) model to test its behavior. In most cases the model can be derived from examining the behavior of the SUT namely from its GUI and can automatically be extracted by ECT via ‘ARA’ the Automated Recording Assistant in Tosca. In some cases the derived model has to be adjusted by the tester to adopt it to special testing aspects (improving the model). The testing approach itself is a “black-box testing” – meaning it is not directly related (and thus depending) on the actual code implementation of the SUT, but observes the behavior on the HMI (Human-Machine-Interface) or GUI (Graphical User Interface). So it detects which data goes into the machine and what results are coming from the machine. Other aspects happening in the coding of the software are not relevant for black-box testing approach.
Model based test automation must not be misunderstood or mixed up with model based testing, which is something completely different. Model based testing means a testing approach where a special (in most cases flow-model) model is derived from the software under test, which is simplified and can be better analyzed in the following process.
The model based test automation approach has lots of advantages over traditional approaches like scipt-based testing:
Value of model based test automation
First of all it makes testing more easy for non-technical personal as it doesn’t require coding or scripting. It is also much more flexible to small program changes in the SUT as it abstracts from the implementation layer to a business layer. And it can be more easily adopted to changes in the SUT and the adopted models can easily be propagated across all depending test cases.
Thus model based test automation is more efficient and reusable compared to other techniques like scripting. It also can achieve high test automation rates of 90% and more.
Risk-based Testcase Design
Another important aspect of ECT is the risk-based testcase design. The focus of software testing should be on the most important and most-at-risk business processes so that time and resources are used efficiently when testing. This means that the parts of the SUT that should be tested have to be chosen according to their business importance and the risk of change under upgrades or updates of the SUT.
Also the test data has to be chosen and selected with respect to risk-based testcase design.
The variation in test data should be as complete as the underlying business process (resp. test model) demands but as compact as possible to perform the task without redundancy.
ECT and CIA play together in this aspect and provide a very efficient and cost effective approach to test automation with respect to most-at-risk business processes.
Three aspects are relevant and are mixed for optimization in risk-based testcase design:
- What to test (CIA derives the most-at-risk processes that should be tested)
- Manual adoption to introduce business importance to ECT
- Test data selection and composition according to risk-based and efficient testing
Value of risk-based testing
Test data Management
Another important aspect of ECT is the included Test data Management (TDM). More precisely this part consists of TDM (Test data management) and TDS (Test data Service). The TDS allows you to easily manage, store, and keep data stateful throughout your end-to-end testing.
TDM additionally allows you to mix test data in advanced combinatorics and generate synthetic test data to allow for most efficient testing with less duplicate testing. TDM can use and mix actual test data recorded from manual transaction processes and synthetically generated data to be most beneficial and efficient in the test automation process with high re-usability rates, thus covering most-at-risk business processes.
Value of Test Data Management
Orchestrated Service Virtualization (OSV)
With OSV you will be able to simulate services in the testing process which might be rarely available for testing in real life due to server outages or involved business impact. OSV is capable of emulating cloud applications and services that might need an external service provider and would put a performance impact on their machines. Thus OSV helps testing services that would otherwise hardly be available for testing.
Value of Service Virtualisation
SAP Enterprise Continuous Testing Integration in SAP Solution Manager, special bundles
SAP Enterprise Continuous Testing integrates with SAP Solution Manager Test Suite and Focused Build in Solution Manager.
SAP Enterprise Continuous Testing comes in two ‘flavors’: the full ECT (functionality as in Tricentis Tosca) or Tricentis Test Automation (TTA) only. TTA is offered free of license charge to SAP customers with Enterprise Support and can be used together with the Test Suite in SAP Solutions Manager to achieve modern test automation. Thus it can replace the CBTA (Component Based Test Automation) in Focused Build, but it needs the Solution Manager to integrate in the overall testing process with test case documentation and results as well as test data management.
The following picture describes the components and dependencies:
ECT versus TTA
The components in grey will be only available in ECT, whereas the part in blue is TTA. For the full testing functionality SAP Solution Manager is needed with TTA to make up for the ‘missing parts’. TTA can only be used for SAP testing, whereas ECT can also be used for all other applications thus allowing for end-to-end tests consisting of multiple technologies (e.g. results are shown in PDF or Excel or an external web application).
ECT and TTA are integrated in SAP Solution Manager and will be integrated in ALM Cloud.
Advice for TTA usage:
TTA is meant for SAP customers that already use the Test Suite in SAP Solution Manager and generally want to stick with that solution but would like to try the test automation in ECT. ECT on the other hand is the more complete bundle with respect to Test Data Management and Provisioning and all the other aspects already mentioned in ECT (grey parts in the picture). A customer can try TTA and switch to ECT later on without losses or redundant tasks.
Nevertheless the workflow will change form SAP Solution Manager with Focused Build + TTA to ECT as many integration parts from TTA to the Solution Manager are no longer needed. For a customer that aims on using the whole test solution in ECT it is better to switch to ECT directly not to lose time on the integration aspect including education of personal.
6. SAP Enterprise Performance Testing by Tricentis (EPT)
In April 2021 Tricentis acquired Neotys which is the leading company in Enterprise Performance Testing Solutions. Neotys product for load and performance testing is NeoLoad and will further on replace “flood”, which used to be Tricentis solution for load testing. On SAP pricelist the product will be offered as SAPEnterprise Performance Testing by Tricentis (EPT) and will be well integrated in ECT.
In general performance testing is as important as functional testing as it will be able to detect software engineering problems already in early development state and will contribute together with ECT/functional testing to overall software stability and reliability. Performance problems even in possibly less-important parts can rapidly add-up to significant amounts thus making a solution unusable for the thought intent or at least less usable and less cost efficient. Functional testing and performance testing should interact together to get the best out of both testing disciplines and make the backbone of agile enterprise continuous development process.
6.1 Key benefits of Enterprise Performance Testing
With EPT you can perform automated API testing with bottleneck identification. Testcases generated in ECT/Tosca can directly be propagated to EPT for performance testing with a single Click. The technology used by EPT fits well for SAP Applications and can be used on the whole SAP technology stack on premise and in the cloud.
Integration between ECT/Tosca and EPT/NeoLoad is tight: whenever testcases in ECT are changed they also will update on EPT if used as performance test:
EPT comes with a modern Dashboard to show all performance parameters and results at a glance. You can drill down to detailed performance results as needed.
SAP Enterprise Performance Testing Architecture
You can deploy EPT central components in the cloud or on premise but there are client components as well that need to be installed on the frontend for the end-user access to performance testing. The load generators can be deployed on premise or in the cloud. Tricentis offers its own cloud service (Neotys cloud) for the central components which can be used without extra cost (included in the license). There is a limitation in cloud usage according to licensed amount of load test users.
7. S/4HANA Migration with Tricentis Support
For most customers the transition to SAP S/4HANA is a challenging task demanding lots of expertise in business processes as well as technical aspects. Business processes managed by SAP run across the whole organization, and even out to other organizations, such as customers, suppliers, or partners. An interruption to any SAP process could potentially have disastrous consequences for everyone involved. Maintaining that business continuity presents a number of significant challenges, both from the business’s perspective and from a technical perspective.
Although the SAP S/4HANA solutions are carefully tested there are still several situations remaining for the customer to test himself:
- Custom Coding
- Third-party Add-ons
- Interfaces
- Solution Extensions
- Native HANA developments
- Integration to web services
It is important to make sure business will run flawlessly in SAP S/4HANA once the transition is done and goes life. Systems don’t stand alone but communicate widely in the corporation and with other partners and suppliers outside. Especially web based services become more and more important and it is obvious that quality assured processes and connections are essential. Tricentis Service Virtualization can help identifying bottlenecks in web connections and multi-customer access as well as analyzing any performance issues.
Furthermore functional and performance tests are important on any interface and connection from and to the SAP S/4HANA system to ensure steady business and reliability. You want to test and quality assure your transition processes as early as possible in each phase to keep costs down and detect issues as early as possible.
8. Additional Information
Tricentis Resources
- White Papers and documentation
- data sheets
- demo videos
- webinars
- Custormer success stories
- Why Tricentis
Tricentis Academy on Youtube
- Tricentis Academy
- Tricentis Tosca: Basic Automation
- Introduction to TOSCA TestSuite
- Tricentis Tosca: TCD TestCase Generation Techniques
- Tricentis TestCase Design Overview
- Tricentis Tosca Enterprise End-to-End Automation
- Introduction to the Tosca SAP Solution Manager Integration(short)
- Tricentis Tosca and Solution Manager Integration:(detailed)
Integration Demo
- Demo SAP 04.07.21 (CIA, ECT, SAP Integration)
- SAP Managed Tags:
3 Comments
