Learn and share on deeper, cross technology development topics such as integration and connectivity, automation, cloud extensibility, developing at scale, and security.
At Basis Technologies our tools are built in ABAP, so abapGit has been on our radar for a while and we’ve been fiddling with it for quite some time (if you’re not familiar with what abapGit is you can learn more here or here). But so far the only ‘productive’ use has been to periodically commit any changes done in development. This has some value for auditing (we can see who changed what and when or compare dev with production, down to the line of code) but it’s a far cry from the holy grail of developing like the rest of the world, with branching and CI pipelines.
However, we recently started the journey towards using abapGit for actual product development. We’ve seen a lot of potential in a Git-based approach for some time but getting it up and running successfully was always going to take some work, so we needed some sort of significant reason to get started. Recently we started planning a large piece of development which will affect most of our codebase and we think that with abapGit we might be able to make it much more efficient, but we are in the early stages. We’ve already learnt a lot but we’ve also hit a few bumps in the road.
Our goal is to adopt for ABAP on premises a workflow similar to what SAP recommends here for mta applications on cloud platform
The first step has been running SAP instances in containers. It’s not absolutely necessary, but since the plan is to give each developer their own personal dev environment, they’re an effective way to reduce costs and simplify onboarding: our system can spawn a brand new instance with about a minute of work and thirty minutes of waiting. For now they’re just a convenient way to quickly spin up and dispose of identical systems.
While not supported by SAP, it does work pretty well, but is quite different to how most people use containers:
Images containing the database have bigger layers than what most registries accept (around 50 GB vs 10 GB for Amazon ECR). You can run your own to overcome the limitation but it’s one more server to manage. We opted for leaving the database in a volume which has advantages but makes creating new instances a bit more complicated
The hostname has to be enforced
Every service needs its own IP, as mapping different instances to different port numbers doesn’t really work
On the bright side none of this is a showstopper when you run it on your own machine. Looking properly into Amazon ECS and similar is on my to do list but what we have is good enough to start with.
Our plan is to:
Develop in private boxes owned by developers
Commit to a git repository, using feature branches
When development is ready
Run as many automated tests as reasonably possible. Go back if failed
Merge on a PC
Import on the developer’s laptop to validate it activates, fixing issues if needed
Pull the merge on a dev system where a transport will be generated
Run some other test/ consistency check
Deploy downstream with transport based tools
Manual QA might sit downstream or after the automated tests.
Last year we demoed a prototype where this merge operation was automated via a Jenkins pipeline, and we definitely want to do that in production in the long run. But the first step is to get this working manually
So we imported our codebase into one box and abapGit serialized to about 13000 files. Then we imported the same code on another, identical box. We will use one as private dev and one as the central dev system that feeds production
First surprise: abapGit found differences between the two as the import process didn’t completely fill the tables abapGit relies on and so it failed to pick up some includes. The ABAP repository tables didn’t list them, and they didn’t show up in SE80 until we reactivated the object, so we reactivated the whole thing on both sides, committed again and finally managed to get the 2 systems in sync.
We imported the code in the second box because pulling from abapGit from scratch doesn’t work. There are too many activation issues, many with circular dependencies. I guess for now we are stuck with transports plus mass activation to kick-start. Switching between branches usually works fine, so we will move forward with our journey towards CI.
We’re going to keep experimenting to see if abapGit can help the way we hope in the large development project we’re planning. The next step will be to do some development in the first box and then generate a transport on the second. This will then be merged on the main development track (we already have a system to detect conflicts with local developments).
I’ll be sharing our progress in a series of blogs like this one, so watch out for more over the next few weeks as we figure out whether abapGit can fulfill the potential we think might be there.
Main issue is that the codebase is pretty big, I don't really know where the issues are, I just get thousands of errors and have no idea what the root cause is. Might try to reproduce on a small scale if I find the time.
For now I will wait until I bump into a smaller case, as soon as we start using this workflow for real they should crop up sooner or later.
Same for the indexes. Happens often in the wild, hard to recreate manually.
use BC sets. Looks like the right thing to do, but we never tried
do that in the TMS landscape
move customising to preprod/prod with transports
test it in preprod
periodically release a 'project customising pack' transport to import in the personal systems
Doubt they will work as smoothly as code but (2) is not as bad as it looks as we use our own software to sequence transports and prevent conflicts between local development and incoming git changes
We plan to do the same, but it's still a pain because it's not a great option for some scenarios, and because seeding with transports forces us to use non local packages which are a pain in the personal boxes
We also plan a small utility to migrate the whole package tree to sort it, we should only need to use it once but don't like doing this sort of thing manually
The idea is that every developer will have their own dev system and git repository, and only be able to affect the main one with pull requests.
And will do most merge/rebase operations on their laptops, connected to both the main repo and the personal one, using their dev system to test the result
I see a lot of challenges, but I think this one is basically sorted
More likely to be an abap program, I see little value in running this out of the box you need it in.
Another one I plan to use for this and will try to get my boss to open source is a code formatter.
Our codebase is not consistently formatted, and while the SAP diff engine is good enough ignoring case and spacing the rest of the world will give you a lot of noise.
Only ABAP tools are not so easy to deliver, test and version (since ABAP platform has different goals and different benefits) - hence, all my tools are written in non-ABAP if possible.