This story explains how to migrate data in Postgres table to elastic search, includes geometry and JSONB data using logstash. Recently I started using Elastic search for our application use case to avoid cross-data joins from microservices.
Logstash helps us in seeding data from different sources to Elastic source indexes. Here, our source is the Postgres DB instance. We have a wide range of use cases one amongst them is migrating geometry and JSONB data to Elastic Search indexes.
I have to spend few hours to crack this, being new to logstash. Spent hours to understand JDBC input plugin and…
This story focuses on a specific index type BRIN (Block Range Index), its purpose, use cases where this is much more efficient than other index types. Also, we draw performance metrics comparing with other popular index types in Postgres.
What is BRIN Index?
A BRIN is a Block Range Index. A block is Postgres’ base unit of storage and is by default 8kB of data. BRIN samples a range of blocks (default 128), storing the location of the first block in the range as well as the minimum and maximum values for all values in those blocks. …
This story talks about securing node js applications from the most common security threats to Node Js application. These are the most common security vulnerabilities that is identified by the OWASP article.
Many of these issues can be addressed by using the Helmet middleware library.
Helmet can help protect your app from some well-known web vulnerabilities by setting HTTP headers appropriately.
Helmet is actually just a collection of smaller middleware functions that set security-related HTTP response headers:
In this story, we will see what is the problem that the bulkhead pattern going to solve. The design patterns are a set of guidelines that we have to follow while building an application for stability, easy maintenance, resilience, etc..
Microservices are distributed in nature. In distributed systems, always remember this rule — anything could happen. We might be dealing with network issues, service unavailability, application slowness, etc. An issue with one system might affect another system's behavior/performance. Dealing with any such unexpected failures/network issues could be difficult to solve.
Ability of the system to recover from such failures and…
This story focuses on Authorizing the API calls with AWS SIGV4 on spring boot applications. This is primarily a use case when we setup AWS API Gateway with IAM Authorizer.
How to do it?
We will use AWS Request Signing interceptor. Basically we use AWS core sdk dependency and customize HTTP Client to make external calls. In the same way we will configure RestTemplate with custom http client to create AWS SIGV4 for every outbound call.
We should include following dependency in our project.
Let us do it in simple Java method first
In this article, we’re going to look at the new Spring Cloud Bus project. Spring Cloud Bus uses lightweight message broker to link distributed system nodes. The primary usage is to broadcast configuration changes or other management information. We can think about it as a distributed Actuator.
The project uses Kafka broker as the transport, but Active MQ or Redis can be utilized instead of Kafka. Other transports are not supported yet.
Before moving forward please look at the spring config server setup that I already documented here. …
This story talks about the encrypting secure data like (passwords, API Keys of third party information ..) at REST when spring cloud config is used as centralized configuration for micro services.
Please read the following blog to get started with Spring cloud config server.
The Config Server can use a symmetric (shared) key or an asymmetric one (RSA key pair). In this story we use both the ways.
For Spring Cloud to be able to encrypt and decrypt properties you will need to add the full-strength JCE to your JVM (it is not included by default). You can download…
This story focuses on composite configuration in spring cloud config with backend as Jdbc (Postgres), Redis and S3. When I am implementing this I didn’t find enough details available on internet with Redis, JDBC and S3. But a lot are available on GIT as backend. So I am not going to talk about GIT here.
High level information about Spring cloud config server
The idea is that our program or programs can move your settings to an external place so that our application is easily configurable and can even change their settings.
This is widely used in microservices. A same…
This story talks about performance of Spring Rest API vs gRPC service implementation. We will use the Apache benchmark to perform a performance test on exhaustive load.
gRPC is an RPC implementation/framework from Google for inter-microservices communication. Google has been using this for more than 15 years (the internal name is Stubby). It is battle tested for more than a decade. Google site shows they have been processing 10 BILLIONS requests / second using gRPC.
We will see why gRPC is faster than REST with a simple. Before that Let see the basic difference between Rest and RPC
This article talks about master-slave configuration set up in local docker setup and we will verify read replica of slave nodes when master is unavailable or down. We can read data of not when the master node is down. This sample is explained with Spring Boot.
In realtime, the application has to process and fetch from multiple tables through joining to get complex data every time. To improve performance, if we could cache the response which does not change frequently, so that performance will be faster.
When cache is implemented there will be a 100% probability that a single instance…
Experience in Open source development, Technical Leader. Expert in Java/J2EE, Integration, analytics. Loves Cricket, cooking, movies and travelling.