SAP AS ABAP Running Inside Docker. PART I

[former_member213255]

 

---

Intro

I think that this article we can consider as a second part about putting SAP products inside Docker container. First one was about SAP HANA in Docker container and it is available via this link. Now let's talk oh how we can put SAP AS ABAP inside Docker. At the end it will give us a complete landscape that we can fire up in several seconds. I will mainly focus on docker stuff, and will not touch orchestration of the containers with Swarm or K8, however you can already find quite a nice blog about putting ABAP AS inside K8.

Technical Overview

This is what we have:

• SAP HANA Docker image


• Custom SLES Image

Nad below is what we want to achieve:

Explanation:

Each SAP system consists of 2 containers: one for the SAP AS ABAP and one for the SAP HANA DB. Since our saphana containers always use the same hostname, we should isolate this containers with bridge adapters - means each system will run in it's own network namespace. This is only relevant in case we want to run more than one dockerised system per docker host.

Such network configuration can be done with the following docker commands:

docker network create -d bridge -o "com.docker.network.bridge.name=sap-br01" -o "com.docker.network.bridge.enable_icc=true" --subnet 172.20.0.0/24 sap-net

docker network create -d bridge -o "com.docker.network.bridge.name=sap-br02" -o "com.docker.network.bridge.enable_icc=true" --subnet 172.30.0.0/24 sap-net02

This will create two docker networks:



From Linux perspective we will have two bridge adapters:

In order to control communication between containers and interfaces, docker will use linux iptables. Linux iptables consist of two tables: filter and nat. In filter table you may find security rules that either allow or deny network communications. 

There are several ways on how we can create SAP AS ABAP container, starting from very simple: Install SAP S/4 instance inside container and than execute docker commit, kind of barbarian way 🙂. This command will take a container's top-level read-write layer and burn it into a read-only layer. Such approach will mutate 🙂 a container (whether running or stopped) into an unchangeable image.

PROS	CONS
Very simple way to create docker image	Image update is complicated

Another way is to prepare Dockerfile, where you should specify instructions, layer by layer:

PROS	CONS
You can update and extend image just by adjusting the Dockerfile. And think on running automated builds	More effort to create image. You should take care about all dependencies

Below we will look on first approach. However for ABAP AS we can combine commit and dockerfile creation way. I will write about this (dockerfile way) approach in next chapter 🙂

---

First Option: "docker commit"

Firstly, we will prepare docker host.

1. Create Docker network


2. Spin Docker Containers with prepared SLES image


3. Download content for the SAP S/4 Installation from marketplace and copy content inside Docker container


4. Run SWPM and install required software


5. Persist data and burn the image

Create Docker Network

This one is  explained few lines above

Spin Docker Containers

• I'm starting SLES 15.1 image from registry:

docker run -i -d --rm --name sapabap01 --network sap-net --ip 172.20.0.2 --ipc=shareable -t -h saphost2 -p 30100-30199:30100-30199 -p 50100-50199:50100-50199 -p 1128-1129:1128-1129 registry.suse.com/suse/sle15:15.1

• Now I'm starting my prepared HANA image from previous blog:

docker run -i -d --rm --name saphana01 --network sap-net --ip 172.20.0.3 --ipc=shareable --memory=30g -t -h saphost1 sap_hana:latest

Now we have 2 containers running in the same network:

Download Content and Copy it to the Container

The easiest way is to run Maintenance Planner transaction. It will create a list of files that are required for the SAP S/4 Foundation installation. You can download then with SAP Download Manager.

As next step I will login to my SLES container, install dependencies from repo and then I will proceed with installation steps (using SWPM). In order to have installation media I need to copy it from the docker host to the docker container:

Run SWPM and Install Required Software

All is now prepared for the SAP S/4 installation. Let's do it.

I will skip whole SWPM process of the system installation, as you should follow the wizard. Below you will find some screenshots that I think can be important. 

1. When you launch SWPM installer you can specify what port should be used to access the installation wizard. That means you can put any mapped port.


2. Once SWPM is started you can access it by using hostname of your docker host and port that was mapped when you start your container.


3.  Proceed with basic SWPM steps. Specify  installation media and HANA client distributive.


4. You will be asked to provide HANA server host name. This is actually our second container - saphana01 (refer to the landscape image in the very beginning of this article)
   


5. You will receive prerequisites check warning. You can ignore it:
   


6. During installation was running, I was hitting the following resource limits:


7. Whole installation took ~15 minutes. At the end I got fully running SAP ABAP S/4 Foundation system distributed between 2 containers:

Persist Data and burn the Image

This is almost last step. System is installed and running. Now it is time to persist data. And this we should do for both containers but in different ways.

For the ABAP we can clean up all installation media and just use docker commit, while for HANA container (since we did not create volumes for SAP HANA DATA files) we should execute full backup (hopefully HANA is doing this very fast) and store it somewhere outside our running container.

Persit ABAP Container

• Clean media folder (BTW it is 9Gb!)


• Clean SWPM installation log from the /tmp (~ 350 MB)


• Uninstall sap host agent  (~ 200 MB)


• Stop all SAP services

Actually we can squeeze it even more. For example /sapmnt can also be deleted, and some other files\folders inside /usr/sap/<SID> (This worth of another article, so I will not discuss it right now)

• Time to commit container into image, by executing:
  docker commit <IMAGE ID> <NEW IMAGE NAME>


• Lets inspect our freshly created image with dive:

Persit HANA Container

We need to backup both tenants of our HANA 2.0. Firstly let see what we have (just to be sure)

SELECT A.DATABASE_NAME, A.HOST, SUM(B.DATA) AS DATA_GB FROM "SYS"."M_VOLUMES_" AS A INNER JOIN (SELECT VOLUME_ID, DATABASE_NAME, ROUND(DATA_SIZE/1024/1024/1024, 3) AS DATA FROM "SYS"."M_VOLUME_SIZES_" WHERE LOG_SIZE =-1) AS B ON A.VOLUME_ID = B.VOLUME_ID AND A.DATABASE_NAME = B.DATABASE_NAME GROUP BY A.DATABASE_NAME, A.HOST;

Ok, so here we have two tenants that we will backup. Meanwhile most of the questions about HANA backups are answered in the following SAP Note 1642148 - FAQ: SAP HANA Database Backup & Recovery.



And then copy from container:

---

Finally now we have:

1. ABAP AS container


2. HANA backup, that we need to restore every time when we start HANA container

However this is not enough. ABAP AS is using hana client and hdbuserstore to communicate with HANA DB server. So, if we will change IP address or container name, or will use another network it will not able to connect to our DB and SAP will not start. That means if we want to start second pair of containers we always should do several adjustments:

1. Recover backup into freshly started SAP HANA container


2. Adjust ABAP default profile and create new hdbuserstore connection in the ABAP container

Recover HANA DB

• Stop Tenant

alter system stop database BFG;

• Recover Tenant

RECOVER DATABASE FOR BFG UNTIL TIMESTAMP '2021-09-21 15:00:00' USING DATA PATH ('/usr/sap/BFG/HDB01/backup/data/DB_BFG/') USING LOG PATH ('/usr/sap/BFG/HDB01/backup/log/DB_BFG/') CLEAR LOG;

•  Start tenant

alter system start database BFG;

Adjust ABAP profile

• Adjust DEFAULT.PFL parameters:

• Recreate hdbuserstore string:

 

Puhhhh, many manual steps to make it fly:



and here is my final picture with four containers running:



 

---

Afterwords

This is my adventure called "Run SAP in Docker". In the very beginning I noticed that there are at least two options how ABAP container can be created. This blog article touches first option. Most probably I will write another article where will I will use second option. 🙂

Future Plans

Firstly, we need to automate as much as possible:

• Recovery procedure


• Adjustment of the abap profile and many other things....

Secondly, we need to prepare docker compose yml files. This will simplify docker run commands.

And then we can think on how to squeeze images.

My target is to create development infrastructure that will allow to do the following things:

• Usage of gCTS instead of old fashion CTS


• Avoid blocking situations (since we can have many development systems)


• Build CI with Jenkins, that will trigger automatic tests

Unfortunately, we still cannot create full landscape without usage of CTS (too many technical architectural limitations), however we can accelerate (speed up) ABAP feature development and do some ABAP version controlling in Git 🙂

 

This is it, thank you for reading 🙂

[UxKjaer]

Really cool, could you create a seperate docker container for the DB for the SAP ABAP systems, so the application servers are separated and therefore we could have multiple application server containers connecting to the same database and use the same data for development?

[former_member213255]

Yes, it is possible to put every SAP service in separate container:

ASCS - docker one
Dialog - docker two
Web Dispatcher - docker three
DB - docker four

With such approach you can start to think about creating swarm\K8 cluster and setup auto scaler...

Regarding data for the development, this is one of the main core issues we have in SAP. ABAP is very different compared to other languages - it is centric, everything is in DB. Objects are in the DB, dictionary... So if you want to have distributed development (working in parallel on the same object), you should provide separate system and this will not work if you will always take copy of prod that usually 500+ GB size, moreover for the development you do not need all master data that is available in prod system (this is historically how ABAP developers used to work). Right approach is either to create mockup data for your development, or export very small portion of master data from the PROD\QA.

 

 

[alejiandro_sensejl]

Wow, this is GOLD! Any chance to have pre-configured docker containers from SAP for AS ABAP developer editions in the future, as MP requires S-User?

[former_member213255]

Hello Alejandro!
I am afraid that this will not happen in near future. Moreover, officially this is not a supported solution (1122387 - Linux: SAP Support in virtualized environments). So to be fair enough I would not recommend to run it for the PRD environments. (Dev might be fine, maybe QA) 🙂

 

[former_member671673]

Docker is the future. Java doesnt dissapoint, fast adoption. ABAP we still waiting for readiness assessment on PRD environments

[efstefst]

Why not simply use plain LXC/LXD?

[former_member213255]

Well this is a matter of discussion. You can if you want, why not.

IMHO: ecosystem is wider for Docker either than for LXC --> so it is much easier to find answers and troubleshoot it.

It is also possible to ask why not to use rocket or something from OCI https://opencontainers.org
This is just a question of your taste and what you already used to work with.
The same kind of discussion can be around what you will select Chef, Ansible, Vagrant, Puppet or something else 🙂

[efstefst]

I was just wondering why you went down the overcomplicated road to create stateful container. But it happens to also when I am learning something, blog and correct it (often months) later 😉

[former_member213255]

Well, as I said this is a topic to discuss. I cannot say that is is overcomplicated :). Depends on what you want to achieve.
1. In the example above it is stateless not stateful container (we do not persist data). This is not the best way for DB however.

2. From my personal opinion Docker is easier to use because of the community - it is much more popular, you have more tools around it and I can have supported OS base images for HANA like SLES or RHEL (did not find the same for LXC)

3. From Architecture point of view, we can say it is almost the same (if I am not wrong) as Docker was grown up from LXC, just additional layers of abstraction on top.

4. This is true,  I did not work a lot with LXC, so Docker was much logical for me as a technology

5. I can create base Image for HANA and than I can adjust it for future dockerfiles, so I have many choices on how I can adjust this image. I can add volumes and make it staful for example....

P.S. You are welcome to share HANA in LXC (not express edition) for the community, that will be very interesting to read (Definitely for me) 😉

[kurtpfingst]

Hi Maxim,

when I'm trying to start the the ./sapinst command, the system sends first

"...extracting ...  done! "

but shortly after that I get the error message

"/tmp/sapinst_exe.273.1615484476/sapinst: line 19: /usr/bin/which: No such file or directory"

 

Do you have any hints/ideas what could be missing here? Have you used the same base_os container as for the HANA 2.0 installation or did you enhance that by any other packages?

 

 

[former_member213255]

What type of distro you are using? Looks like you are missing which utility in this image. Can be installed with zypper.

 

[sandeepkarnati]

Hello Afonin,

Thank you very much for sharing this.

how do i bring the current development system to Containers ?

do i need to create a blank container and backup current Dev system and Restore it in Container?

Thanks

[former_member213255]

Hello sandeepkarnati !

This is one of the options, probably the easiest one. Before doing any kind of restore you should clean it up from non used data in order to make container as small as possible.

 

Another possible option that from my point og view more nice is to have empty DEV system container. Every time you need DEV system you can spin it up, than on top od this system you can move your development (transport based or gCTS) this even can be automated. All test data need to be generated, transfarred based on the development case. Such approach is much more complex, but it is more flexible.

[sandeepkarnati]

Hello Afonin,

Thank you for the response.

lets say i choose 1st option ( restoring the current DEV system on to a container).

now lets say,i have ECC and CRM in our landscape.which means i have to spin 2 containers (ECC and CRM) from each developer,correct ?

if thats the case,if ECC and CRM need to communicate each other, how do i point the RFC's in ECC and CRM containers ?

or if ECC and CRM need to communicate external systems,how do i configure RFC's in  this scenario on container SAP?

 

Thanks

 

[filipecruz]

Hi Maxim! This post is fantastic!!

I just completed the OpenSAP course on gCTS and came across your blog post while doing some research on docker.

You mentioned: "Unfortunately, we still cannot create full landscape without usage of CTS (too many technical architectural limitations), however we can accelerate (speed up) ABAP feature development and do some ABAP version controlling in Git 🙂"

Do you know if you still would face the same limitations today compared when you created this post?

Thanks!!

Kind Regards,

Filipe

[former_member213255]

Hello filipecruz

We still have the same limitations just because of:
ABAP architecture do not allow as to decouple execution environment from development environment. As a result, the whole idea on how this developments are distributed impose certain technical restrictions. Yes, for sure gCTS is brings distributed development much closer rather than it was before. But there are still certain things that are not well resolved (we cannot natively and effectively use git approaches during development like rebase or cherry pick technics.

For sure it is possible to find a workaround solution, but the effort of such implementation especially on the big project will strike out the whole positive effect.

And one more thing, git usage will require another approach in the ABAP development, means you should push your developers for changes. Most of the people do not like to change 😊

P.S. I still would recommend this as an innovation project for some feature development in the last S/4 releases with not big dev team where you can try to implement\play around with pipelines, automatic testing, and other nice things from the non-SAP IT world 😊)

[filipecruz]

Thanks maxim.afonin !

I was just curious especially after watching Week 5 - Unit 6 video where it shows "Old Process" vs "New" with each ABAP developer having its own Dev. system using Docker - That would be really cool!! But it does not explain much on how to achieve this. Then I ended up here and saw your comments regarding technical limitations..

Yes! I'm very much interested in the gCTS, maybe as a small POC first. I am an ABAP developer for quite some time, but also used other technologies and Git and always wanted to have the same in ABAP world. But I totally agree with you that it would require drastic changes to the ABAP dev. process and heavy training starting on Git.. unit testing, CI/CD tools etc. The first person to implement this would be hated for some time I guess 😄

 

Thanks again for the excellent post and I will keep watching for new updates on gCTS and ABAP with CI/CD 🙂

 

Kind Regards,

Filipe

[AhmedKanoun]

Hi,

i get this error : /tmp/sapinst_exe.121.1698402782/sapinst: line 19: /usr/bin/which: No such file or directory
no such file: ./sapinstexe

 

how i can solve it