Linux

awk Command Examples

awk is a powerful text processing language. Here are common use cases with examples:

Basic Syntax

awk 'pattern { action }' input_file

1. Print Specific Columns

# Print first and third columns
awk '{print $1, $3}' file.txt

# Print last column using NF (Number of Fields)
awk '{print $NF}' file.txt

2. Filter Rows by Condition

# Lines where 3rd column > 100
awk '$3 > 100 {print $0}' data.csv

# Lines containing "error"
awk '/error/' log.txt

# Case-insensitive match
awk 'BEGIN{IGNORECASE=1} /warning/' system.log

3. Field Separators

# Use comma as separator (CSV processing)
awk -F',' '{print $2}' data.csv

# Multiple separators (space or comma)
awk -F'[ ,]' '{print $3}' mixed_data.txt

# Change output separator
awk -F':' -v OFS='|' '{print $1, $3}' /etc/passwd

4. Calculations & Variables

# Sum values in 4th column
awk '{sum += $4} END {print "Total:", sum}' sales.dat

# Average of 2nd column
awk '{sum += $2; count++} END {print "Average:", sum/count}' numbers.txt

# Using variables
awk -v threshold=50 '$3 > threshold {print $1}' readings.txt

5. Built-in Variables

# Print line number with content
awk '{print NR, $0}' access.log

# Process only first 10 lines
awk 'NR <= 10' large_file.txt

# Skip header row
awk 'NR > 1 {print $2}' header_data.csv

6. String Manipulation

# Extract first 3 characters
awk '{print substr($1, 1, 3)}' words.txt

# String concatenation
awk '{print $1 "-" $2}' parts.txt

# Replace text (similar to sed)
awk '{gsub(/old/, "new"); print}' document.txt

7. BEGIN and END Blocks

# Add header and footer
awk 'BEGIN {print "Report\n------"} 
     {print} 
     END {print "------\nEnd of Report"}' data.txt

# Initialize variables
awk 'BEGIN {FS=":"; OFS="\t"} {print $1, $3}' /etc/passwd

8. Regular Expressions

# Lines starting with 200
awk '/^200/' status_codes.log

# Lines with 4-digit numbers
awk '/[0-9]{4}/' mixed_content.txt

# Negative match (exclude pattern)
awk '!/test/' experimental_data.txt

9. Conditional Logic

# If-else statement
awk '{if ($2 > 50) print "High"; else print "Low"}' values.txt

# Multiple conditions
awk '$3 == "USA" && $4 > 1000' sales_data.csv

10. Advanced Usage

# Process multiple files
awk '{print FILENAME, NR, $0}' file1.txt file2.txt

# External commands
awk '/ERROR/ {system("echo " $0 " >> errors.log")}' app.log

# Custom functions
awk '
function double(x) { return x*2 }
{ print $1, double($2) }
' numbers.txt

Common Options

  • -F - Set field separator
  • -v - Define variables
  • -f - Read program from file
  • -i inplace - Edit files in-place (GNU awk)