Category Archives: Tips and Tricks

Bash Shell – Folder as tree

Leave a reply

I like using tree command in Linux, thankfully it has been ported to Mac as well and it is quite easy to get tree representation of files and folders inside a directory on a shell. An article on StackOverFlow talks about it aptly. The original site is here.

In case you are using brew then it is quite easy peasy as shown below:

brew install tree

The following commands generate a tree view:

Generates a tree of all files and folders in the current directory:

tree

Generates a tree containing only folders in the current directory:

tree -d

Generates a tree containing only folders for 3 levels depth. 

tree -d -L 2 .

In case you want to use vanilla shell without installing anything. (doesn’t generate a tree though).

find . -maxdepth 3 - type d

Throttling & Tuning Spring Boot

Leave a reply

One of my Spring-Boot projects was battling with overloaded CPU and unresponsive / slow server response at times when there is more traffic. I have explored and implemented caching but my problem was with excessive connections coming in and server itself becoming slow. I could have setup multiple instances and do some kind of auto-scaling but given limited budget and hardware I wanted to put in some hard limits on my Spring-Boot app as to how much traffic it can take in and when it can give up gracefully (there is no shame in rejecting traffic with HTTP status 503 if the server infrastructure is overloaded).

I found a blog post entry from Netflix on how to tune Apache Tomcat and another article on how to tune Rest Controller code itself to implement a rudimentary Rate Limiter. I was glad to find the RateLimiter implementation in Goggle Guava library which I ultimately ended up using (for now). However I think the annotation driven RateLimiter is also a very good solution which is certainly very powerful and I will take it out for a spin sometime in near future.

The basic lesson learnt from this exercise:
– Tweak Tomcat and keep a watch on the acceptCount parameter which technically puts in a limit of how much traffic reaches your Rest controller.
– Use a RateLimiter on your hot APIs (which have higher latency) and don’t let your application get abused beyond a limit.
– Scale horizontally if the limits set above result in lot of traffic getting rejected.

Block a shell script till a server boots up

Leave a reply

This small script blocks a shell script / docker compose command script till a dependent server boots up.

#!/usr/bin/env bash

while :
do
  response=$(curl --write-out %{http_code} --silent --output /dev/null -X GET "$1")
  if [[ "$response" -ne 200 ]] ; then
    echo "Server is not yet up >> $1 >> $response"
    sleep 1
  else
    echo "Server is up >> $1 >> $response"
    exit 1
  fi
done

Taking backup zip from multi-module Java project

Leave a reply

I like to keep an archive of my code on a monthly basis. I wrote a small bash script to create zip out of my multi module Java projects. It gets the job done and can be improved to include resources and test folders as well. As of now I am only interested in the Java code zip backup.

!/bin/bash
 if [ -d "combined" ]; then
     echo "Removing existing zip files in combined .."
     rm combined/*.zip
 else
     echo "Creating combined folder.."
     mkdir combined
 fi
 for d in */ ; do
     if [ -d "$d/src/main/java" ]; then
         var="$(echo $d | sed 's/.$//')"
         echo "Processing $var .."
         cd $d/src/main/java
         zip -r ../../../../combined/$var.zip com
         cd ../../../..
     else
         echo "Ignoring .. $d"
     fi
 done

ssh_exchange_identification: read: Connection reset by peer

2 Replies

I recently upgraded my Mac Desktop and I noticed a strange problem that I was not able to SSH into my desktop from my laptop. I ultimately found out that the SSH setup that came with MacOS was having some issue as when I did ssh user@localhost on the desktop I got the same error:

ssh_exchange_identification: read: Connection reset by peer

I ended up tailing the system log using the following command:

sudo tail -f /var/log/system.log

I observed the following entries in the log whenever a SSH was attempted. 

com.apple.xpc.launchd[1] (com.openssh.sshd.[UUID][NUM]): Service exited with abnormal code: 1

It was clear that there was setup issue with the SSH that came with default MacOS installation. So I decided to spawn a separate SSH instance and watch it’s log:

sudo /usr/sbin/sshd -d -p 2222

This command showed up a lot of issues related to file permissions.

Permissions 0644 for '/etc/ssh/ssh_host_dsa_key' are too open.
Permissions 0644 for '/etc/ssh/ssh_host_ecdsa_key' are too open.
Permissions 0644 for '/etc/ssh/ssh_host_ed25519_key' are too open.

I fixed these permission issues by changing their permission to 400:

sudo chmod 400 /etc/ssh/ssh_host_dsa_key
sudo chmod 400 /etc/ssh/ssh_host_ecdsa_key
sudo chmod 400 /etc/ssh/ssh_host_ed25519_key

After this change the following command succeeded and I was able to do successful SSH connection to port 2222.

sudo /usr/sbin/sshd -d -p 2222

So I killed this process and decided to restart SSH:

sudo launchctl unload /System/Library/LaunchDaemons/ssh.plist 
sudo lsof -i:22
echo $?
sudo launchctl load /System/Library/LaunchDaemons/ssh.plist 

Once SSHD was restarted I could successfully do logins using ssh user@localhost from Desktop as well as remote login via SSH from my laptop. 

Prettify JSON on Terminal

Leave a reply

I love using curl command on my Mac terminal to debug my REST endpoints. However the REST call JSON output used to come in a blob of text which required further formatting in an Editor like Visual Studio Code. To allay this problem I ended up installing “jsonpp” using homebrew.

brew install jsonpp

So now I just pipe the output of my curl command to the jsonpp program and I get a fully formatted JSON.

$ curl http://localhost:8080/test | jsonpp
{
"year": 2018,
"month": 2,
"worked": 18,
"leaves": 2
}