Mastering Linux Shell Scripting

Table of Contents

Mastering Linux Shell Scripting

Credits

About the Author

About the Reviewer

www.PacktPub.com

Support files, eBooks, discount offers, and more

Why subscribe?

Free access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Downloading the color images of this book

Errata

Piracy

Questions

1. What and Why of Scripting with Bash

Bash vulnerabilities

The bash command hierarchy

Command type

Command PATH

Preparing text editors for scripting

Configuring vim

Configuring nano

Configuring gedit

Creating and executing scripts

Hello World!

Executing the script

Checking the exit status

Ensuring a unique name

Hello Dolly!

Running the script with arguments

The importance of correct quotes

Printing the script name

Debugging your scripts

Summary

2. Creating Interactive Scripts

Using echo with options

Basic script using read

Script comments

Enhancing scripts with read prompts

Limiting the number of entered characters

Controlling the visibility of entered text

Enhancing learning with simple scripts

Backing-up with scripts

Connecting to a server

Version 1 – ping

Version 2 – SSH

Version 3 – MySQL/MariaDB

Summary

3. Conditions Attached

Simple decision paths using command-line lists

Verifying user input with lists

Using the test shell builtin

Testing strings

Testing integers

Testing file types

Creating conditional statements using if

Extending if with else

More conditions with elif

Creating the backup2.sh using elif

Using case statements

Script – building a front-end with grep

Summary

4. Creating Code Snippets

Abbreviations

Using code snippets

Bringing color to the terminal

Summary

5. Alternative Syntax

Recapping test

Testing files

Adding logic

Square brackets as not seen before

Providing parameter defaults

Variables

Special parameters

Setting defaults

When in doubt – Quote!

Advanced test using [[

Whitespace

Other advanced features

Pattern matching

Regular expressions

Regular expression script

Arithmetic operations using ((

Simple math

Parameter manipulation

Standard arithmetic tests

Summary

6. Iterating with Loops

For loops

Controlling the loop

While loops and until loops

Reading input from files

Creating operator menus

Summary

7. Creating Building Blocks with Functions

Introducing functions

Passing parameters to functions

Returning values from functions

Using functions in menus

Summary

8. Introducing sed

Using grep to display text

Displaying received data on an interface

Displaying user account data

Listing the number of CPUs in a system

Parsing CSV files

The CSV file

Isolating catalog entries

Using regular expressions

Working with alternate spellings

How many words have four consecutive vowels?

RE anchors

Understanding the basics of sed

Substituting command

Editing the file

Summary

9. Automating Apache Virtual Hosts

Apache name-based Virtual Hosts

Creating the Virtual Hosts template

First steps

Isolating lines

sed script files

Automating Virtual Host creation

Prompting for data during site creation

Summary

10. Awk Fundamentals

The history behind awk

Displaying and filtering content from files

Formatting output

Further filtering to display users by UID

Awk control files

Summary

11. Summarizing Logs with Awk

The HTTPD log file format

Displaying data from web logs

Selecting entries by date

Summarizing 404 errors

Summarizing HTTP access codes

Displaying the highest ranking IP address

Displaying the browser data

Working with e-mail logs

Summary

12. A Better lastlog with Awk

Using awk ranges to exclude data

The lastlog command

Horizontal filtering rows with awk

Counting matched rows

Conditions based on the number of fields

Manipulating the awk record separator to report on XML data

Apache Virtual Hosts

XML catalog

Summary

13. Using Perl as a Bash Scripting Alternative

What is Perl?

Hello World

Perl arrays

Program name?

Argument arrays

Counting elements in an array

Looping through an array

Creating arrays

Conditional statements in Perl

Replacing command line lists

If and unless

Using functions within Perl

Prompt for user input

Creating the function

Summary

14. Using Python as a Bash Scripting Alternative

What is Python?

Saying Hello World the Python way

Pythonic arguments

Supplying arguments

Counting arguments

Significant whitespace

Reading user input

Using Python to write to files

Summary

Index

Mastering Linux Shell Scripting

Mastering Linux Shell Scripting

Copyright © 2015 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: December 2015

Production reference: 1171215

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham B3 2PB, UK.

ISBN 978-1-78439-697-8

www.packtpub.com

Credits

Author

Andrew Mallett

Reviewer

Peter Ducai

Commissioning Editor

Kartikey Pandey

Acquisition Editors

Kirk D'costa

Harsha Bharwani

Content Development Editor

Mayur Pawanikar

Technical Editor

Saurabh Malhotra

Copy Editor

Sneha Singh

Project Coordinator

Nidhi Joshi

Proofreader

Safis Editing

Indexer

Hemangini Bari

Production Coordinator

Shantanu N. Zagade

Cover Work

Shantanu N. Zagade

About the Author

Andrew Mallett is the owner of The Urban Penguin and is a comprehensive provider for professional Linux software development, training, and services. Having always been a command-line fan, he feels that so much time can be saved through knowing command-line shortcuts and scripting. TheUrbanPenguin YouTube channel, maintained by Andrew, has well over 800 videos to support this and he has authored four other Packt titles.

About the Reviewer

Peter Ducai has worked for more than 15 years within the IT industry, which includes programming in C, Java, Python, administration, and troubleshooting OS (mainly Unix/Linux), and also testing and automation.

Currently working as a contractor for HP, he specializes in network automation.

He has worked on the book, Learning Embedded Linux Using the Yocto Project.

I want to thank Becca for making my busy life easier.

www.PacktPub.com

Support files, eBooks, discount offers, and more

For support files and downloads related to your book, please visit www.PacktPub.com.

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.

https://www2.packtpub.com/books/subscription/packtlib

Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library. Here, you can search, access, and read Packt's entire library of books.

Why subscribe?

Fully searchable across every book published by Packt

Copy and paste, print, and bookmark content

On demand and accessible via a web browser

Free access for Packt account holders

If you have an account with Packt at www.PacktPub.com, you can use this to access PacktLib today and view 9 entirely free books. Simply use your login credentials for immediate access.

Preface

Mastering Linux Shell Scripting will become your Bible and a handbook to create and edit bash shell scripts in Linux, OS X, or Unix. Starting with the fundamentals, we quickly move onto helping you create useful scripts with practical examples. In this way, your learning becomes effective and quick. With each chapter, we provide explanations of the code and code examples, so from a learning book this becomes a book that you can use as a ready reference, if you need to understand how to program a specific task.

What this book covers

Chapter 1, What and Why of Scripting with Bash, explains how to create and name scripts. Once you have the script created you can make it executable and welcome yourself to the world. If you have little or no knowledge of scripting then you can start here.

Chapter 2, Creating Interactive Scripts, covers the scripts we will need to work in a more flexible manner and accept arguments or even prompt the user for input during the script execution. I am sure that you have seen scripts similar to this asking for installation directories or server's tp connect to.

Chapter 3, Conditions Attached, covers the use of keywords, such as if, and commands like test. It tells us how we can start creating decision structures in the code and then prompt the user for input, if we have not provided arguments; otherwise, we can run silently.

Chapter 4, Creating Code Snippets, covers the vim text editor, which is very powerful and also syntax highlighting to help us edit the script. However, we can also read into files of the current script. In this way, we can create snippets of code to represent commonly used blocks.

Chapter 5, Alternative Syntax, tells us how we can abbreviate the test command to just a single [, we can also use [[ and (( depending on your needs.

Chapter 6, Iterating with Loops, covers how loops are also conditional statements. We can repeat a block of code while a condition is true or false. In this way, using for, while, or until we can have the script complete the repetitive code sequences.

Chapter 7, Creating Building Blocks with Functions, covers how functions can encapsulate the code that we need to repeat within the script. This can help with readability and how easy a script is to maintain.

Chapter 8, Introducing sed, the stream editor, tells us how sed can be used to edit files dynamically and implement it in scripts. In this chapter, we look at how to use and work with sed.

Chapter 9, Automating Apache Virtual Hosts, covers the practical recipes that we can take away when we create a script to create virtual hosts on an Apache HTTPD server. We use sed within the scripts to edit the template used to define virtual hosts.

Chapter 10, Awk Fundamentals, looks at how we can start to process text date from the command line and using awk is another very powerful tool in Linux.

Chapter 11, Summarizing Logs with Awk, tells us about the first practical example we look at with awk, allowing us to process log files on the web server. It also looks at how to report the IP address that has access to the server most often, as well as, how many errors occur and of which type.

Chapter 12, A Better lastlog with Awk, looks at more examples that we can use in awk to filter and format data provided by the lastlog command. It drills down to the specific information that we want and removes information we do not need.

Chapter 13, Using Perl as a Bash Scripting Alternative, introduces the Perl scripting language and the advantages that it can offer. We are not restricted to just using bash we also have Perl as a scripting language.

Chapter 14, Using Python as a Bash Scripting Alternative, introduces you to Python and the Zen of Python that will help you with all programming languages. Like Perl, Python is a scripting language that can extend the functionality of your scripts.

What you need for this book

Using any Linux distribution with the bash shell should be sufficient to complete this book. In the book we use examples that are produced using the Raspbian distribution on a Raspberry Pi; however, Linux distribution should be sufficient. If you have access to the command line in OS X on Apple systems, then you should be able to carry out most of the exercises without Linux.

Who this book is for

Mastering Linux Shell Scripting has been written for Linux administrators who want to automate tasks in their daily lives, saving time and effort. You need to have command-line experience and be familiar with the tasks that you need to automate. A basic knowledge of scripting is expected.

Conventions

In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.

Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: We again see that the basename is evaluated first, but we do not see the more detailed steps involved in running that command.

A block of code is set as follows:

#!/bin/bash

echo You are using $0

echo Hello $*

exit 0

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

#!/bin/bash echo You are using $0

echo Hello $*

exit 0

Any command-line input or output is written as follows:

$ bash -x $HOME/bin/hello2.sh fred

New terms and important words are shown in bold. Words that you see on the screen, for example, in menus or dialog boxes, appear in the text like this: Clicking the Next button moves you to the next screen.

Note

Warnings or important notes appear in a box like this.

Tip

Tips and tricks appear like this.

Reader feedback

Feedback from