Author Archives: cyberaka

About cyberaka

I am an experienced Senior Solution Architect with proven history of designing robust and highly available Java based solutions. I am skilled in architecting designing and developing scalable, highly available, fault tolerant and concurrent systems which can serve high volume traffic. I have hands on experience in designing RESTful MicroServices architecture using Spring Boot, Spring Cloud, MongoDB, Java 8, Redis and Kafka. I like TDD (JUnit), BDD (Cucumber) and DDD based development as required for my projects. I have used AWS primarily for cloud based deployment and I like developing cloud enabled POC as a hobby in my spare time. I have deigned and developed CI/CD pipeline using Jenkins and leveraged Docker, Kubernetes for containerizing and deploying some of my applications. I am highly experienced in creating high performing technical teams from scratch. As an ex-entrepreneur I am very much involved in the business side of the IT industry. I love interacting with clients to understand their requirements and get the job done.

Duplicating a Git repository

Leave a reply

I was working in a personal Git repository and I wanted to make a duplicate copy of it under a different organisation. The following approach worked well for me:

Open Terminal / Command Prompt and Clone the existing source repo:
git clone --bare https://your_source_repository_url

Make sure you don’t skip the “–bare” portion of the command line.
You will see that a new folder is created from your source repository.


Move inside the folder:
cd your_source_repository

Now we need to push the files from your source repository into your target repository:
git push --mirror https://target_repository_url

Make sure your don’t skip the “–mirror” portion of the command line. Also this command has to be executed from the source repository’s clone location folder.

Localhost Tunnels

Leave a reply

I have started using ngrok for setting up localhost tunnels directly from laptop. The basic idea is to start a web server in localhost and use ngrok to setup a tunnel to internet. This is very easy way to test local code and get it validated with other team mates. I will be looking out for an enterprise offering for this wonderful tool.

A very simple use case of ngrok is demonstrated in this video which is a tutorial for webhooks.

Bash Shell – Folder as tree

Leave a reply

I like using tree command in Linux, thankfully it has been ported to Mac as well and it is quite easy to get tree representation of files and folders inside a directory on a shell. An article on StackOverFlow talks about it aptly. The original site is here.

In case you are using brew then it is quite easy peasy as shown below:

brew install tree

The following commands generate a tree view:

Generates a tree of all files and folders in the current directory:

tree

Generates a tree containing only folders in the current directory:

tree -d

Generates a tree containing only folders for 3 levels depth. 

tree -d -L 2 .

In case you want to use vanilla shell without installing anything. (doesn’t generate a tree though).

find . -maxdepth 3 - type d

Code Structure Analysis Tool

Leave a reply

I received a dump of Java codebase which had multiple modules and I needed to analyse it’s structure. The following tool did quite a good job:

https://github.com/gdela/socomo

Basically the idea is to run it inside a Java project using a maven command and it creates HTML file which denotes the high level structure of the code.

Note: Stan4J is also a very good tool which does similar job but allows deeper analysis (upto 500 classes only)

Mongodump – Unrecognized field ‘snapshot’

Leave a reply

I recently upgraded my mongo instance on my laptop and I was trying to do a backup of my mongo database on a remote server. I tried using my usual monodump commands but saw some strange error:

Failed: error reading collection: Failed to parse: { find: "Files", skip: 0, snapshot: true, $readPreference: { mode: "secondaryPreferred" }, $db: "app_db" }. Unrecognized field 'snapshot'.

On some googling I found out that this error can happen due to difference between the Mongo server version and the Mongo client version. This stackoverflow article explains the problem and the solution to it as well.

In the end I simply had to add “–forceTableScan” to my mongodump command to get it to work properly.

Throttling & Tuning Spring Boot

Leave a reply

One of my Spring-Boot projects was battling with overloaded CPU and unresponsive / slow server response at times when there is more traffic. I have explored and implemented caching but my problem was with excessive connections coming in and server itself becoming slow. I could have setup multiple instances and do some kind of auto-scaling but given limited budget and hardware I wanted to put in some hard limits on my Spring-Boot app as to how much traffic it can take in and when it can give up gracefully (there is no shame in rejecting traffic with HTTP status 503 if the server infrastructure is overloaded).

I found a blog post entry from Netflix on how to tune Apache Tomcat and another article on how to tune Rest Controller code itself to implement a rudimentary Rate Limiter. I was glad to find the RateLimiter implementation in Goggle Guava library which I ultimately ended up using (for now). However I think the annotation driven RateLimiter is also a very good solution which is certainly very powerful and I will take it out for a spin sometime in near future.

The basic lesson learnt from this exercise:
– Tweak Tomcat and keep a watch on the acceptCount parameter which technically puts in a limit of how much traffic reaches your Rest controller.
– Use a RateLimiter on your hot APIs (which have higher latency) and don’t let your application get abused beyond a limit.
– Scale horizontally if the limits set above result in lot of traffic getting rejected.

Nginx gzip compression and load balancing.

Leave a reply

To tune performance of my REST endpoints in past I have enabled Gzip compression in my nginx server configuration. So technically a large json response becomes gzipped and the network latency as a result goes down.

There is a good documentation of this feature on the nginx website which does a pretty good job.

However there is a catch which prevents this technique from working on local development system (while the same config works in production linux instance). I finally found an answer as to why this doesn’t work in some of my local environment.

To do static load balancing I use the upstream concept of nginx which is documented again on the nginx website. The performance is reasonable and the implementation is quite simple for a requirement which needs a simple failover implementation. However for advanced implementation we can always go to haproxy which is very good open source load balancer.

Issues Faced While Upgrading to JDK 11 and Spring Boot 2.2.5

Leave a reply

I had a old project which was running on JDK 8 and Spring Boot 1.5.x which I just upgraded to JDK 11 and Spring Boot 2.2.5. I faced some hiccups in the process but it is finally done and the application is up and running. I will be documenting some of the issues faced in this process.

Spring Data
All instances of findOne() had to be replaced with findByID() which returns Optional<Entity> reference. I ended up removing lots of null checks by leveraging the Optional features of “orElse”, “orThrow” and “ifPresent” methods provided by Optional interface. It definitely is a better way to handle null.

JAXB runtime error
After migration the application booted up fine but on doing some specific operation I observed this error on console

java.lang.ClassNotFoundException: javax.xml.bind.DatatypeConverter

It seems this class has been moved out of the core library in JDK. This error kind of stumped me as some posts on internet suggested adding the following entry in pom.xml

<dependency>
   <groupId>javax.xml.bind</groupId>
   <artifactId>jaxb-api</artifactId>
   <version>2.3.1</version>
   <scope>runtime</scope>
</dependency>

However this didn’t help and I ended up using the following library to finally get rid of this issue:

<dependency>
    <groupId>jakarta.xml.bind</groupId>
    <artifactId>jakarta.xml.bind-api</artifactId>
</dependency>

Apache POI / Tika Library
This is another runtime issue faced during this application operation. 

WARNING: An illegal reflective access operation has occurred
 WARNING: Illegal reflective access by org.apache.poi.openxml4j.util.ZipSecureFile$1 (file:/Users/562320/.m2/repository/org/apache/tika/tika-app/1.15/tika-app-1.15.jar) to field java.io.FilterInputStream.in
 WARNING: Please consider reporting this to the maintainers of org.apache.poi.openxml4j.util.ZipSecureFile$1
 WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
 WARNING: All illegal access operations will be denied in a future release
 Jun 29, 2020 3:14:17 AM org.apache.catalina.core.StandardWrapperValve invoke
 SEVERE: Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.apache.poi.openxml4j.exceptions.OpenXML4JRuntimeException: Fail to save: an error occurs while saving the package : class org.apache.poi.openxml4j.util.ZipSecureFile$ThresholdInputStream cannot be cast to class java.util.zip.ZipFile$ZipFileInputStream (org.apache.poi.openxml4j.util.ZipSecureFile$ThresholdInputStream is in unnamed module of loader 'app'; java.util.zip.ZipFile$ZipFileInputStream is in module java.base of loader 'bootstrap')] with root cause
 java.lang.ClassCastException: class org.apache.poi.openxml4j.util.ZipSecureFile$ThresholdInputStream cannot be cast to class java.util.zip.ZipFile$ZipFileInputStream (org.apache.poi.openxml4j.util.ZipSecureFile$ThresholdInputStream is in unnamed module of loader 'app'; java.util.zip.ZipFile$ZipFileInputStream is in module java.base of loader 'bootstrap')
     at java.base/java.util.zip.ZipFile$ZipFileInflaterInputStream.available(ZipFile.java:480)

This issue got resolved after upgrading the Apache Tika, Apache POI to latest version. I also had to upgrade the apache-commons-lang package to apache-commons-lang3 package.