Author Archives: cyberaka

About cyberaka

I am an experienced Senior Solution Architect with proven history of designing robust and highly available Java based solutions. I am skilled in architecting designing and developing scalable, highly available, fault tolerant and concurrent systems which can serve high volume traffic. I have hands on experience in designing RESTful MicroServices architecture using Spring Boot, Spring Cloud, MongoDB, Java 8, Redis and Kafka. I like TDD (JUnit), BDD (Cucumber) and DDD based development as required for my projects. I have used AWS primarily for cloud based deployment and I like developing cloud enabled POC as a hobby in my spare time. I have deigned and developed CI/CD pipeline using Jenkins and leveraged Docker, Kubernetes for containerizing and deploying some of my applications. I am highly experienced in creating high performing technical teams from scratch. As an ex-entrepreneur I am very much involved in the business side of the IT industry. I love interacting with clients to understand their requirements and get the job done.

JSON to Java Code Generator

3 Replies

I recently received a couple of REST Web Service endpoints where the implementation technology was not Java. So I had to either rollout my own object and then rely on the object mapper to map the JSON to Java or come up with an approach to generate the relevant POJO file. On a little bit of looking around I found out the following two links for generating Java Code from JSON.

http://json2java.azurewebsites.net/

https://javafromjson.dashingrocket.com/

The POJO generated were decent and got the job done. I didn’t have to write the POJO myself and the code generated by these websites did the job well.

Another link recommended in the posts comments.
http://www.freecodeformat.com/json2pojo.php (Thanks Daniel)

I found out the 1st two links are no longer working. So I found out about another tool based on Visual Studio Code. Use the following extension for solving this use case.

https://marketplace.visualstudio.com/items?itemName=quicktype.quicktype

Delete Untracked File Using Git

Leave a reply

Sometimes we want to clean out our Git workspace and remove any file which is untracked. A hard reset usually does the job for modified files as shown below:
git reset --hard

But hard reset doesn’t fit in case of untracked files. The following command will do the job:
git clean -d -fx ""

-x means ignored files are also removed as well as files unknown to git.

-d means remove untracked directories in addition to untracked files.

-f is required to force it to run.

Reference:
https://www.kernel.org/pub/software/scm/git/docs/git-clean.html

Leveraging Tee and Grep for filtering program output

Leave a reply

I have a utility which compiles my UI code and minifies them. It works pretty well and doesn’t requires much supervision. Trouble starts if the code compilation fails and it gets ignored in the huge log file which is generated by this utility. I use the following approach to filter out errors and yet preserving the output of the utility program for any diagnostic in case of any error.

./my_utility.sh | tee output.txt | grep error

Basically what happens is that “my_utility.sh” generates a lot of output which is piped into the “tee” command which in turn dumps it into a text file named “output.txt” and then the same output is again piped into the “grep” command which looks for any error text in the output. If the error is found it is outputted on the console.

So the end result is that in 99% cases I don’t see any output from the utility as I don’t want to see debug output. However the moment an error happens it gets flagged on console and I have the “output.txt” file to audit what went wrong and the root cause of the error.

Configuring Git user name and email

Leave a reply

Since I like working using terminal I prefer to do all my operations of git from the command prompt. The first prerequisite to this is to have my name and email configured. This few commands are required to do this simple operation:

To set the name and email:

git config --global user.name "Your Name"
git config --global user.email "Your Email"

To view the name and email which is known to git execute the same commands without any parameter as shown below:

git config --global user.name
git config --global user.email

Finding class inside a bunch of jar

Leave a reply

Many times there are some Java linkage errors and I have to find out in which jar files the class files are located. So this has lead me to find out tools which can do this job for me. I usually get the job done by using this excellent open source tools named Jar Explorer inside Github. It is basically a platform independent Swing based utility which allows you to recursively search inside Jar files located inside a folder for any class name String. So it is possible for me to search for a class named “LoggingEvent” inside a folder containing lots of jars and it outputs the list of all the jar files where it found classes containing the text “LoggingEvent”.

However when you are connected to Linux consoles using ssh and don’t have access to X Windowing system then you have to rely on either text based java program or pure vanilla shell scripting. For this situation I use the following snippets of code which I found from a Stackoverflow article.

On Linux/Mac
for i in *.jar; do jar -tvf "$i" | grep -Hsi ClassName && echo "$i"; done

On Windows
for /R %G in (*.jar) do @jar -tvf "%G" | find "ClassName" > NUL && echo %G

Using tput to label tmux panes

1 Reply

As I had pointed out in my post about tmux that now I am using tmux to configure and debug multiple servers inside a single window split into panes. Now I ran to another problem I always forgot which pane was meant for which service. Beyond 2 to 3 panes it was getting confusing to remember which pane is monitoring which service. So I remembered my previous post related to tput which allows anybody to show a running clock inside a linux terminal. So I decided to provision a small shell script to fix this issue. Basically I wanted a way to label each pane so that I could effortlessly identify the purpose of the pane inside tmux. So here is the code:

#!/bin/bash
#Display Service Name

function die {
echo "Dying on signal $1"
exit 0
}

function redraw {
local width length;
width=$(tput cols);
str=$1;
length=${#str}+10;
tput sc;
tput cup 0 $((width-length));
set_foreground=$(tput setaf 7)
set_background=$(tput setab 1)
echo -n $set_background$set_foreground
printf ' Service:%s ' $str
tput sgr0;
tput rc;
}

trap 'die "SIGINT"' SIGINT
trap 'die "SIGQUIT"' SIGQUIT

trap redraw WINCH;

while true; do
redraw $*;
sleep 1;
done

This shell script takes a parameter and shows it on the top right column in a red background with white foreground. This script should be invoked in this way.

./ShowTextInTerm.sh service-name &

Invoking this script in every tmux pane with relevant substitution for service-name is giving me this result:
Screen Shot 2016-02-11 at 1.59.55 pm

Please note that this script works well on my MacBook. I am yet to test it on a Linux terminal.

Using SSH to clone Git repository with multiple private keys

Leave a reply

I prefer to use the multiple private keys and avoid using the same private key with multiple services. Recently I decided to switch from HTTPS based Git clone of my bitbucket repositories to SSH based Git clone. So created a new private key added them into bitbucket.org and then I expected that my git clone would work. But it didn’t.

git clone git@bitbucket.org:mybitbucketid/mygitrepository.git
Cloning into 'mygitrepository'...
conq: repository access denied.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

I knew I had to point the git client to my private key so I created the following entry in my ~/.ssh/config file.

Host bitbucketrepo
HostName bitbucket.org
IdentityFile ~/.ssh/bitbucket_private_key
User git

Now I felt it would work. I tried again and it still didn’t work.

git clone bitbucketrepo:mybitbucketid/mygitrepository.git
Cloning into 'mygitrepository'...
conq: repository access denied.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Now I was confused as to why it was not working. On further research I found this link. Now I updated the entry in my ~/.ssh/config file to this.

Host bitbucketrepo
HostName bitbucket.org
IdentityFile ~/.ssh/bitbucket_private_key
IdentitiesOnly yes
User git

Now I tried again. This time it worked perfectly.

git clone bitbucketrepo:mybitbucketid/mygitrepository.git
Cloning into 'mygitrepository'...
warning: You appear to have cloned an empty repository.
Checking connectivity... done.

Well the addition of “IdentitiesOnly yes” line in my config file did the trick. It seems that when we do an SSH connection it’s default behavior is to send the identity file matching the default filename for each protocol. So if you have a file named ~/.ssh/id_rsa then that will get tried before your private key which in my case was ~/.ssh/bitbucket_private_key. So by using the “IdentitiesOnly yes” line I explicitly asked my ssh client to use my identity file and nothing else and it worked like a charm.

Awesome tmux

1 Reply

I have always found moving between tabbed interfaces cumbersome in my Mac terminal. Finally when debugging 7 different log files sitting on 7 different servers I felt enough was enough and started looking for a solution. Enter ‘tmux’.

tmux is a terminal multiplexer which supports multiple windows inside a single terminal session and allows me to create horizontal as well as vertical panes. So I can debug the logs sitting on different servers by tailing them in different pane and then still can have one pane dedicated for executing commands. I found a very good tutorial at this and this location.

Now I don’t want to go back to the old way of having multiple tabs for multiple logs. Agreed tabs in terminal have their own place and usage but for this particular case where I am doing development and debugging multiple servers I don’t have time for a tabbed interface instead ‘tmux’ is the way to go.

Fixing “Write failed: Broken pipe” on Yosemite

Leave a reply

After latest update to my macbook pro I noticed that my SSH connections started dropping if I kept them idle for few minutes. Each time the session terminated with the text “Write failed: Broken pipe”. I have observed that the connection used to hang for a long while before this error message was shown.

This was not the behavior before so I suspected that recent updates might have changed some configuration and hence I started looking around. I found an article which explained how to configure my Macbook.

Based on the inputs provided in the article I edited the file /etc/ssh_config using the following command:
sudo vi /etc/ssh_config

And changed/uncommented the following lines:
Host *
ServerAliveInterval 60
TCPKeepAlive yes

I learnt another point in this article that during the SSH session if I press “~” followed by “.” then the connection terminates immediately. If it doesn’t then pressing enter before doing this helps.

MySQL Slowdown With Large Inserts

1 Reply

On a plain vanilla windows system with approximately 6 GB of RAM and a XAMPP based MySQL installation I found out that as the number of inserts increased the MySQL inserts became slower and slower. Ultimately it came down to one insert in 2 seconds!

This was totally unacceptable so I looked around for some solution to this problem and I found one here.

Based on the article above I started looking into the MySQL configuration and I found the following entries:

#innodb_log_arch_dir = "D:/xampp/mysql/data"
## You can set .._buffer_pool_size up to 50 - 80 %
## of RAM but beware of setting memory usage too high
innodb_buffer_pool_size = 16M
innodb_additional_mem_pool_size = 2M
## Set .._log_file_size to 25 % of buffer pool size
innodb_log_file_size = 5M
innodb_log_buffer_size = 8M
innodb_flush_log_at_trx_commit = 1
innodb_lock_wait_timeout = 50

So my database is using innodb file system but the buffer pool size is only 16 MB with additional increase of 2 MB. This in my opinion is too less keeping in mind the comments provided in the file.

## You can set .._buffer_pool_size up to 50 - 80 %
## of RAM but beware of setting memory usage too high

So in a 6 GB RAM system I could easily set 5 GB as the buffer pool size but it is a development system and I wanted the developer to have decent development performance as well. So I tweaked the configuration a little bit and now it looks like this:

#innodb_log_arch_dir = "D:/xampp/mysql/data"
## You can set .._buffer_pool_size up to 50 - 80 %
## of RAM but beware of setting memory usage too high
innodb_buffer_pool_size = 1000M
innodb_additional_mem_pool_size = 250M
## Set .._log_file_size to 25 % of buffer pool size
innodb_log_file_size = 50M
innodb_log_buffer_size = 80M
innodb_flush_log_at_trx_commit = 1
innodb_lock_wait_timeout = 50

At the moment the innodb_buffer_pool_size is 1000 MB with an additional innodb_additional_mem_pool_size of 250 MB.

I also increased the innodb_log_file_size to 50 MB and innodb_log_buffer_size to 80 MB (I simply multiplied the default value with 10).

I have started the inserts again into the same table using the same program with no other changes and I see a marked improvement in performance. The insert operation has not yet slowed down. I am able to insert 10-14 records within one second.

Update after 3 hours =====
I observed that after inserting close to 110000 records the inserts slowed down to 4-5 inserts per second. I have read an article which summarizes how InnoDB actually works. I think I will tweaking the configuration a little bit more.