Angle Grinder - Tool to Format and Parse Log Files in Linux

Are you bored opening files in different text editor while working on command line to parse and gain only information that is required. Writing longer commands to slice only required data, or regex commands for formatting. Let's solve your problem using ag (Angle-Grinder) tool. Yes, ag lets you Slice, format & parse files on the command line.

Processing Millions of row in second or analysing data from file at lightning speed can only be possible using Angle grinder. While processing any files results are live updated on terminal which lets you control your data and you can stop process whenever required. Parsing a document and retrieving needful information from json, log or any other files made easy on the terminal. For the most part, you don't have to think about how software works, and you get to focus on the interactive software and use the terminal in the way you want. This tool is built for purpose of making terminal more powerful and user can efficiently utilise its time rather wasting it on other non-terminal tools.

Features of Angle Grinder :

Parse, Aggregate, sum & sort data in files

Live updates on terminal

Better data analytics - logs checking

Perhaps the biggest advantage of ag over traditional text processing methods for looking through logfiles is that it makes it easy to search by date (not covered in tutorial)

So without wasting more time, let's get our hand dirty.

Installation

agrind/ag/angle-grinder tool is supported on any version of Linux and also on Mac OS. One single command on any system and ag is ready for you to use.

Installation takes around 2-3 min depending on your internet speed.
Once you install ag tool, all you have to do is check if "ag or agrind" command are working on your machine.

Syntax

agrind tool works with many series of operator and multiple "|" for passing output of each part to filter or parse or aggregate data. Generally, initial operation will parse data files or JSON followed by aggregation or grouping or fining total no of occurrences this list is not limited you can explore your self and learn different ways for other operators as well.

Using ag tool

we are using sample files available in package for demonstration, and for future commands don't forget this syntax,

agrind ' | operator1 | operator2 | operator3 | ...'

so let's start with different example one by one,

Filter operator

Filter command will list down lines with ERROR tag in them, lets say that you are working with Network file and all of sudden you need to find out error from its log. This simple command will work like charm for you. All you need to do is pass WORD inside ''(single quote).

$ sudo cat filter_test.log | agrind '"ERROR"'

Non-Aggrgate Operator

The test_json log file has error messages. This agrinder query will show all lines because we used *. And It'll show output as JSON (JavaScript Object Notation) format. i.e. as lightweight data-interchange format. It'll display each data from every log file lines in square brackets. Means human readable. Data is splitter now into value (Key) and message (Value),

$ cat test_json.log | agrind '* | json'

Where

where if(condition) is satisfied, row is kept as is but if the condition is not satisfied rows where dropped. In the command below left is column name while right is the condition. Examples

About Yash Vasa

Yash, a passionate Linux System Administrator and tech enthusiasts. He drive immerse gratification in conducting research on Industrial IoT platforms and keeping up to date with the latest technology world. He believe that combination of brilliant mind, gold ideas & utter disregards for what is possible can & will change the world.

Related Posts

As of recently, a cryptocurrency craze has swept Internet communities. The people have found out that one does not need a several thousand dollar-worth rig to mine cryptocurrency with a profit. Stores are short on graphics cards, and a gold [...]

VNC stands for Virtual Network Computing. The VNC server allows you to view and access a computer desktop remotely over the network. In Linux, the VNC server is used to display an X session on some other system. This article [...]

Hi everyone, today we'll learn how we can interactively create a docker container using a docker image. Once we start a process in Docker from an Image, Docker fetches the image and its Parent Image, and repeats the process until it reaches [...]