Continuous Integration and Deployment (CI/CD) for Oracle SOA SuitePublished on: Author: Richard Velden Category: Oracle
In this blog, I will elaborate on several considerations when implementing CI/CD for Oracle SOA Suite. This is by no means, a complete CI/CD guide for Oracle SOA Suite. However, I do intend to show what kind of choices one can make. For many choices, there are benefits and tradeoffs. Almost every pro has a con, and it is up to you to decide what cons to take for granted.
The practice of CI is to integrate each developer’s code continuously (at least daily) into the main code repository. This needs to be done to prevent so-called merge-hell. CI also entails building the code continuously (at each check-in), deploying the new version, and running automated unit tests to ensure quality and stability of the product. CD takes this even further by continuously deploying code to production. To do so one needs to automate the deployment process all the way up to production while ensuring software quality.
The benefit to developers is huge. By automating builds, deployments, tests and the release process, we get more time doing fun stuff. Let us not forget earlier bug detection, which frees us from debugging on Test environments.
So, what choices to make when implementing CI/CD? In particular what choices to make for an Oracle SOA Suite environment. In the following chapters, I will discuss several aspects of CI/CD in relation to Oracle SOA Suite.
In Oracle SOA Suite, every separate composite is deployable independently. Therefore, every change on composite A can be build and deployed separately of all other components.
- Q: “But what if some composite B depends on composite A?”
- A: “If you don’t break the interface in A you don’t need to change, build and redeploy B right? And if you need to break the interface, you need to change B as well, which automatically builds and deploys B anyway.”
- Q: “But redeploying A should also trigger unit tests of dependent composites such as B.”
- A: “Good point, but what are the implications?”
Take for instance some generic document service. Almost all composites rely on this service either directly or indirectly. Changing this generic document service would force us to run almost all of our unit tests. Certainly, we can make this work, but it forces us to keep track of all dependencies between SOA composites.
However, keeping track of things takes time and discipline. Alternatively, running an ‘All Tests’ job every night could serve the same purpose, albeit with a little delay. Ensuring no dependent composites were hurt during the day. I am a lazy person, so you may guess which option I recommend.
Single Build Plan compared to Build Plan for each Composite
Let me just start by saying that this ignited discussion at my previous project.
Build Plan for each Composite
Many consider this the most agile way of employing build plans. For each composite, you create a single build plan in your CI/CD tool of choice.
- The CI/CD Console shows the state of all builds per composite. If a single composite is broken, you can find it here.
- One broken composite does not interfere other builds.
- It enables special build steps for some composites.
- Many similar build plans.
- Every new composite requires creating/cloning a build plan, and configuring it for another project.
- Harder to modify plan: adding features to the ‘build plan’, forces you to change all build plans. Usage of shared scripts can alleviate this problem. Some CI tools even offer build plan templates, which can help as well.
Single Generic Build Plan
A naive single build is one that just builds all composites. However, I am talking about a more flexible option, almost as flexible as the ‘build plan per composite’. Using the commit details, one can use a simple script to make sure the generic build only compiles the changed composites. Each source code repository commit triggers this generic build, and each commit contains all the changed composites. All SOA composites compile and deploy the same, so a single generic script/build should be flexible enough.
- No need to maintain multiple builds plans which are in fact all the same. Changes to the build plan are easy, where it only requires modifying a single plan.
- Adding new composites does not imply creating or changing build plans.
- It is not clear whether there are still broken composites based on the state of the generic build plan.
E.g.: The first build failed on composite A, but the next build for composite B succeeded. A successful build does not imply the bugs from previous builds are fixed. One could also choose to let all next builds fail until the initial issue (composite A) is fixed. However, this makes developers, each working on separate projects, too dependent on each other. It is thus up to the developer to check his/her build, fix the issues, and commit a new version soon. Running an ‘All Tests’ job every night helps to ensure developers fix their commits daily.
We chose for the single generic build at my previous assignment, partly because something similar was already in place. However, the most important reason for me was the lack of proper template functionality in our CI/CD tooling (Bamboo). In a perfect world, a proper build plan template could enable us to change the template, which in turn automatically changes all de template based build plans. Every build plan change implies a lot of boring, manual, repeating, and error prone work without such a mechanism. Just for rolling out a small change through all plans.
Having many similar build plans, without proper templates, greatly reduces developer willingness to add features or refactor the build plans. In my opinion, this is the main reason not to choose for such a solution.
Componentization and Branching
For SOA Suite, it is less usual to have a release of all composites with version 1.x. Each composite is versioned independently. Bear in mind that for many monolithic applications the entire source needs to be build and deployed as a whole. Even for more componentized applications, multiple dependencies with other internal components and (external) libraries are required for a successful build.
In an Oracle SOA Suite landscape, composites are largely independent. There is no big monolith, and each service compiles and deploys independently. In a way, one could model every composite to have its own version control, separate of all the others.
This takes us to branching. There are many ways of using source control, and branching is an important feature. What choices to make depends on your situation. Some very good resources on the subject can be found [1,2].
Development on Feature Branch
Every new feature or change starts with creating a feature branch. One creates a feature branch from the master branch, to many known as the trunk (see figure 1). Developing on a feature branch allows developers a certain level of isolation from the main branch. The drawback being just that: isolation from other developer’s changes.
According to some, this defeats the purpose of continuous integration altogether. By postponing integration to the Master branch, merge conflicts arise more often. Moreover, building and unit testing on the Master branch happens less often. Building and testing on the separate feature branches can solve part of this problem. Amusingly, this practice has been coined as Continuous Isolation instead of Integration.
Feature branching does serve a purpose in some use-cases:
- It allows feature developers flexibility and independence of other feature branches. Especially for features that take long to complete, and which span multiple releases.
- It allows a separate team to control the Master branch. This team can do code reviews and separate testing. Only stable features, which adhere to the coding standards, will merge to Master.
A common model where feature branching can help is with larger open source projects. It provides flexibility to feature teams, but also provides a model for the primary maintainers to enforce some standards on the mainline.
Development on trunk
All developers commit their changes directly into the trunk, as depicted in figure 3. Preferably at least daily. They explicitly do not commit directly to release branches!
We create a tag or branch from the trunk to release code. This release branch can then be hardened and tested more rigorously. Developers can continue to develop new features and commit these into the trunk during this process. Any bugs found in this release branch are first fixed on the trunk. Finally, the change is cherry picked from the trunk and merged into the release. Fixing bugs on the release branch first sounds easier. However, chances are one forgets merging the bug fix back to the trunk. Surprisingly, the bug re-occurs in the next release.
Our Twist on Trunk Based Development
In our project, we chose to do trunk-based development with a twist. Developers still only commit in the trunk, where every commit triggers the CI build/deploy and tests. Our release branch however, starts out with the production state. This is actually just a tag with the previous release number. Our twist: All changes deemed fit for rolling through other environments are cherry picked from the trunk into the release branch. On each commit, we build, deploy and test the release branch on a separate environment. On success, we push this particular release commit onto the next environment, until it finally reaches production.
Maven and Nexus
There are several blogs on how to do Nexus and Maven with Oracle SOA Suite. However, I have not seen one that explains why one should use Nexus or Maven for OracleSOA. Especially when one can also use ANT, which is more flexible. And ANT is my friend 😊! In a nutshell:
Why use Maven?
- It imposes a standardized project structure. Developers accustomed to Maven can easily understand the overall build structure.
- Dependency management: External libraries / Internal components.
- Can be configured to use Nexus as a central repository manager.
Why use Nexus?
- Centralized storage of libraries and caching of libraries from external repositories.
- Centralized storage of build artifacts (in our case compiled composites).
Maven and SOA
You can define dependencies using Maven for each component (in the .pom file). Dependencies are either on external libraries, or on other components.
This is a very useful feature for Java projects. Maven standardizes the way developers specify dependencies on external libraries and between components. A Maven build actually triggers the download of libraries and other dependencies before a component is compiled and build into an artifact.
Maven looks a bit less useful for SOA Suite. SOA composites generally do not need external libraries, and do not depend on each other during compile. For building composites, Maven dependencies are kind of misused to download the SOA compiler into its local repository (around 500MB+ of jar files). This can be downloaded either from the Oracle Maven repository, or directly from one of your own middleware homes using the Oracle Maven synchronization plugin.
Nexus and SOA
A centralized repository to cache external libraries and store internal dependencies really helps for Java projects. Java components actually depend on other libraries and components. However, in case of Oracle SOA Suite, there is no real use for this dependency management. In my opinion, reducing Nexus to just a big file repository for SOA artifacts.
I do think it’s still OK to use Nexus or Maven in an Oracle SOA landscape. But in this landscape, the real discerning benefits of these tools are not fully utilized.
 Continuous delivery workflows with the branch-per-issue model, by SARAH GOFF-DUPONT