Introduction
Multi target applications (or MTA) are great. They allow us to package and fully describe an entire solution comprising of frontend applications, backend services (e.g. CAP services), database artefacts, required cloud services, etc. all in a declarative manner in a single, readable file.
You're probably using them already as they're used by default in SAP project templates. For example, when you start a new CAP project it comes with an mta.yaml.
With SAP's mbt tool you can package the mta app into a deployable mtar file and deploy that into the cloud fairly easily.
All services are created and linked between themselves for you. This makes it much harder to have issues caused by updating certain parts of the live system without updating others.
The problem
One downside of using mta files and the mbt tool though is that build times can get very long.
It is not uncommon for your typical CAP project to take 10 minutes to build. All it takes is having many simple frontend apps, an approach that is natural if using Fiori elements for developing apps.
A common solution to this is to split your project into multiple mtas, maybe one mta for frontend and one for backend, or an mta per "function area" but in general that is something I avoid if possible as it negates the advantages of having mta files in the first place. I.e. forcing different inter-dependent components to be updated together.
So how can we reduce the build time of large MTA apps? In this post I'll present two approaches but would love to know of others. Please share in the comments below!
Parallelization
When you right click on the mta.yaml file of your project in BAS and choose "Build MTA Project",
Menu
BAS is actually only running a single command:
mbt build -s '/home/user/projects/xyz-project'; sleep 2;You might see this appear in the terminal at the top of the build process and if we run that command on the terminal, we'd start a build. Perfectly equivalent to the mouse right click option.
The mbt build tool is documented here and has a few additional flags. An interesting one is -j. . This option configures the number of
Make jobs that can run simultaneously. It requires -m (mode) set to verbose. The full command would be something like this:mbt build -s '/home/user/projects/xyz-project' -m=verbose -j=8; sleep 2;What this does is the same build as before but starts multiple jobs at the same time. Instead of building one module at a time, sequentially, it builds J at the same time.
If your project has many modules (.e.g many frontend apps ) it can build up to 8 of those frontend applications at the same time.
The maximum number is 8. In an ideal world this would cut your project's build time to 1/8th. However in practice you may get closer to 1/2 or 1/4 of the original build time.
There are 2 things I don't love about this approach. One is that the output of the build jobs gets intermingled so its nearly impossible to read when a build issue occurs. The other is that this option is flagged as BETA in the documentation.
In my experience I've never seen it fail but you never know if it will produce a broken build. I use this when building locally in BAS but never when doing builds in a CI system.
Being lazy
The limitations of the -j option above got me thinking a bit deeper about the build process and how everything is packaged into a single mtar in a typical node.js CAP project. In general:
- Each module is built independently - each ui app goes into its dist/app.zip file
- All built modules are packaged in an mtar file
In the case of your typical CAP project with many UI apps the largest contributor to the overall build time are the UI applications, and in particular doing an
npm install for every single one of them.However, in the vast majority of situations when we're doing a build, our apps haven't changed. Maybe one or two have changed but not the majority of them. So why is the tool rebuilding every single app instead of using the intermediate results of the previous build?
The nice thing is we can customise how to build apps in an mta project.
In the mta.yaml we go from the default:
- name: mygreatuiapplication
type: html5
path: app/my_great_ui_application
build-parameters:
build-result: dist
builder: custom
commands:
- npm install
- npm run build:cf
supported-platforms: []to a custom script
- name: mygreatuiapplication
type: html5
path: app/my_great_ui_application
build-parameters:
build-result: dist
builder: custom
commands:
- ../build.sh
supported-platforms: []What is this build.sh cuustom script? This shell script will be created in the
/app folder of the project and it will:- calculate the most recently changed source file in that application
- calculate the create/change date of the packaged application zip file
- Perform npm install/npm run build only if the source files are more recent than the zip file that may exist in the dist folder.
Because of the above observation that in most builds only a small subset of files/apps have changes, this allows us to skip build for most apps, reducing the build time drastically.
Using this approach we see typical build times going down from 10minutes to around 30 seconds.
Note: I'm leaving build.sh as an interesting (BA)SH exercise to the reader π Here are a few tips:
- get the name of the most recently changed file perhaps using
find -type fand the-printfflag to also print changed timestamps. - Use sort/tail/cut or some awk magic to get the most recently modificated file from the above list.
- Use bash
if [A -nt B]to see if file A is "newer than" B.
Which techniques do you use to speed up MTA builds? Please share in the comments! Thanks
Take care
- SAP Managed Tags:
6 Comments
