Category Archives: Miscellaneous

Converting Chrome HAR file to CSV

Leave a reply

It is sometimes easier to export a chrome har file into CSV format so that it can be opened in Excel. Once opened in Excel it is really easy to do further analysis for rather very large har dumps.

The following solution is based on this Link: https://stackoverflow.com/questions/31648078/how-to-import-har-file-to-excel

Install jq on Mac. 

brew install jq

Run this command.

cat some_file.har | jq '[ "URL", "Time", "Wait time", "Status", "Body size","Content-Type", "Content-Encoding"],
    (.log.entries[] | [
        .request.url,
        .time,
        .timings.wait,
        .response.status,
        .response.content.size,
        .response.content.mimeType,
        .response.content.encoding
]) | @csv' | sed 's/\\"//g' | sed 's/"//g' > some_file.csv

This results in a CSV file which you can easily open in an Excel.

To extract the path of the URL minus the file name the following Excel function can be used by adding a column:

=LEFT(A2,LEN(A2)-LEN(RIGHT(A2,LEN(A2)-FIND("@",SUBSTITUTE(A2,"/","@",LEN(A2)-LEN(SUBSTITUTE(A2,"/",""))),1))))

Nginx gzip compression and load balancing.

Leave a reply

To tune performance of my REST endpoints in past I have enabled Gzip compression in my nginx server configuration. So technically a large json response becomes gzipped and the network latency as a result goes down.

There is a good documentation of this feature on the nginx website which does a pretty good job.

However there is a catch which prevents this technique from working on local development system (while the same config works in production linux instance). I finally found an answer as to why this doesn’t work in some of my local environment.

To do static load balancing I use the upstream concept of nginx which is documented again on the nginx website. The performance is reasonable and the implementation is quite simple for a requirement which needs a simple failover implementation. However for advanced implementation we can always go to haproxy which is very good open source load balancer.

Adding docker support in Spring Boot application

Leave a reply

Adding docker containerization support in a working Spring Boot application is surprisingly easy.

Step 1: Add the docker maven plugin to your pom file.

<build>
    <plugins>
        <plugin>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-maven-plugin</artifactId>
        </plugin>
        <plugin>
            <groupId>com.spotify</groupId>
            <artifactId>dockerfile-maven-plugin</artifactId>
            <version>1.3.4</version>
            <configuration>
                <repository>${project.artifactId}</repository>
            </configuration>
        </plugin>
    </plugins>
</build>

Step 2: Build the application as well as the docker image

mvn clean package install dockerfile:build

Step 3: I have a local docker registry setup in my Lan so I prefer to push this image to my network registry manually (this can be automated as well however).

docker tag my-app:latest my_network_registry_host:5000/my-app

Step 4: Now we can bootup this docker image anywhere in the network.

docker run \
--detach \
--link <any database link> \
-e <Environment variable 1> \
-e <Environment variable 2> \
-p 8080:8080 \
--name my-app \
my_network_registry_host:5000/simply-hr
 Step 5: Checkout the application URL @ http://localhost:8080. Tail the log if you want to
docker logs --follow simply-hr

Setting up Redmine docker instance

Leave a reply

In order to setup a fully functional Redmine server via Docker I use the following consolidated command:

docker run \
--detach \
--name redmine \
--link mysql:mysql \
--publish 3000:3000 \
--env="REDMINE_DB_MYSQL: redmine" \
--env="REDMINE_DB_PASSWORD: redmine" \
--volume /Volumes/DATA/Abhinav/Docker/redmine/files:/usr/src/redmine/files \
redmine

This connects my mysql docker instance with sonar and it is able to create tables and start using MySQL properly for storing all it’s data.

Setting up Sonarqube docker instance

Leave a reply

In order to setup a fully functional Sonarqube server via docker I use the following consolidated command:


docker run \
--detach \
--name=sonarqube \
--publish 9000:9000 \
--publish 9092:9092 \
--env="SONARQUBE_JDBC_USERNAME=sonar" \
--env="SONARQUBE_JDBC_PASSWORD=sonar" \
--env="SONARQUBE_JDBC_URL=jdbc:mysql://mysql:3306/sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true" \
--link mysql:mysql \
sonarqube:5.1

This connects my dockerized mysql instance with sonar and it is able to create tables and start using MySQL properly for storing all it’s data.

Setting up MySQL docker instance

Leave a reply

In order to setup a fully functional MySQL server via docker I use the following consolidated command:

docker run \
--detach \
--name=mysql \
--env="MYSQL_ROOT_PASSWORD=abcd1234" \
--publish 6603:3306 \
--volume=~/Docker/mysql/conf.d:/etc/mysql/conf.d \
--volume=~/Docker/mysql/mysql-datadir:/var/lib/mysql \
mysql

The advantage of the above approach is that the configuration files can be defined under ~/Docker/mysql/conf.d folder on my system and it will be picked up whenever I bootup this instance. The second thing is that all information written by this instance will be stored outside of VM in the path ~/Docker/mysql/mysql-datadir on my system. So In case this VM goes away due to some unexplained reasons I can still bootup another docker instance of MySQL and point it to this data directory.