Batch DNS inquiries stall

13 replies [Last post]
amenex
Offline
Joined: 01/04/2015

After preparing a script to perform a large number of dig -x inquiries one-at-a-time I'm finding
that it stalls every so often. I'm forced to reconnect the WiFi to nudge the script back into
operation. Is there a command that I can insert in the script to perform the reconnect periodically
so I don't have to babysit the script ? It takes about an hour normally.

amenex
Offline
Joined: 01/04/2015

Answering my own question:
https://askubuntu.com/questions/811733/how-can-i-restart-my-wi-fi-connection-from-the-command-line where it says:
nmcli radio wifi off && sleep .05s && nmcli radio wifi on
Requires apt-get install wl
The original link says "sleep 5" but I find that "sleep .05s" doesn't slow down the script much at all.

Update: Once incorporated in the script, the wifi off & on command doesn't let the dig -x command start and update the output.
It has to be made conditional in the event of a stall.
Here is my so-far-unsuccessful attempt:
count0=0 \
dig -x 100.24.191.100 | grep -A 16 "ANSWER SECTION" | sed 's/;; ANSWER SECTION://g' | awk 'NF=5 {print $5,$1}' | grep "in-addr.arpa" '-' >> A-16C.txt | count=<(wc -l A-16C.txt | awk '{print $1}') ; if count>$count0 \ then $count0=count \ else nmcli radio wifi off && sleep 0.05s && nmcli radio wifi on \ fi ;
dig -x 100.24.191.101 | grep -A 16 "ANSWER SECTION" | sed 's/;; ANSWER SECTION://g' | awk 'NF=5 {print $5,$1}' | grep "in-addr.arpa" '-' >> A-16C.txt | count=<(wc -l A-16C.txt | awk '{print $1}') ; if count>$count0 \ then $count0=count \ else nmcli radio wifi off && sleep 0.05s && nmcli radio wifi on \ fi ;

The dig -x and wc -l sections are functional and the wifi on / off script cycles the wifi but the variable assignments aren't working.

Magic Banana

I am a member!

Offline
Joined: 07/24/2010

Your command lines make little sense, with an output that is at the same time redirected with >> and through a pipe and with a > that is a redirection although you want a comparison.

To run 'nmcli radio wifi off && sleep 0.05s && nmcli radio wifi on' if A-16C.txt was not modified during the last second (for instance), you can write:
#!/bin/sh
sync A-16C.txt
expr $(date +%s) - $(stat -c %Y A-16C.txt) \> 1 && nmcli radio wifi off && sleep 0.05s && nmcli radio wifi

You can put that in an infinite loop (along with a sleep, to not waste CPU resources) or in a crontab (maybe executed every second) or you can create a systemd timer.

amenex
Offline
Joined: 01/04/2015

My interpretation of Magic Banana's syntax appears to be OK as there aren't any bash complaints forthcoming,
but there seems to be a timing issue: the wifi turns off & on quite rapidly ... and then stays off.
The DNS lookup, text processing & posting take less than 0.05 second:
time dig -x 100.24.191.100 | grep -A 16 "ANSWER SECTION" | sed 's/;; ANSWER SECTION://g' | awk 'NF=5 {print $5,$1}' | sed 's/\.0\/24//g' >> A-16MB.txt
That's way less than the \> 1 second threshold for the rattle-the-cage loop:
; while : ; do sync A-16MB.txt ; expr $(date +%s) - $(stat -c %Y A-16MB.txt) \> 1 && nmcli radio wifi off && sleep 0.5s && nmcli radio wifi on; sleep 0.5s; done
The first hundred or so IPv4 addresses shouldn't trigger that step, but when the script starts,
there's a few 0's and then many 1's in the standard output, because the wifi stays off after that.
However, the output file is getting written only once before I have to stop the script. Commenting
out the nmcli script after the first ten DNS lookups does ensure that the output file is written
as many times as there are comments, but then the wifi stays off after a few 1's appear.
It appears that
nmcli radio wifi off && sleep .05s && nmcli radio wifi on
disables the wifi rather than simply disconnecting the wifi; can that be changed ?

The sed script that I interjected in the DNS lookup gets rid of the ".0/24" that interferes with the extraction
of the IPv4 address from the canonical name that is noted in the following Forum comment.

Magic Banana

I am a member!

Offline
Joined: 07/24/2010

My interpretation of Magic Banana's syntax appears to be OK as there aren't any bash complaints forthcoming, but there seems to be a timing issue: the wifi turns off & on quite rapidly ... and then stays off.

Is 'sleep 0.05' enough? I do not know much about networking. I do not think it makes much sense to worry about saving a few seconds whenever the script stalls (what is exceptional, right?): try giving 5 as an argument of both sleep commands.

The first hundred or so IPv4 addresses shouldn't trigger that step, but when the script starts, there's a few 0's and then many 1's in the standard output, because the wifi stays off after that.

Those 0 and 1 (that you can silence redirecting the standard output to /dev/null) are the results of expr's evaluations: 0 if A-16MB.txt was recently modified, 1 otherwise. For each 1 that is output, 'nmcli radio wifi off && sleep .05s && nmcli radio wifi on' (or whatever you write instead of that) is executed.

disables the wifi rather than simply disconnecting the wifi; can that be changed ?

Taking a look at nmcli's manual, it looks like you want to use 'nmcli connection down ID' and 'nmcli connection up ID'.

amenex
Offline
Joined: 01/04/2015

Magic Banana came up with this way of performing wifi disconnects:
Taking a look at nmcli's manual, it looks like you want to use 'nmcli connection down ID' and 'nmcli connection up ID'.

It's never that simple:
nmcli -t -f NAME c show --active
Reply (on a new line):NNNN5 1

There are several issues:
(1) The name of the connection has an embedded [forbidden] space; $NAME has to be a variable, as Issue (3) indicates.
(2) The command replies on a new line; does that disrupt the script ?
(3) My wifi name can be any one of these: "NNNN5", "NNNN5 5", or "NNNN5 1" so the evaluation has to be performed on every line of the script.

Partial solution is to enclose the embedded-space variable in double quotes and to dump all the 0's & 1's results:
dig -x 100.24.191.100 | grep -A 16 "ANSWER SECTION" | sed 's/;; ANSWER SECTION://g' | awk 'NF=5 {print $5,$1}' | sed 's/\.0\/24//g' >> A-16MB.txt ; while : ; do sync A-16MB.txt ; expr $(date +%s) - $(stat -c %Y A-16MB.txt) > /dev/null \> 60s && nmcli connection down "BLBK5 1" && sleep 5s && nmcli connection up "BLBK5 1"; sleep 0.5s; done ;
Output:
Connection 'BLBK5 1' successfully deactivated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/4)
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/5)

... and the singular output of dig -x was successfully printed.

Magic Banana

I am a member!

Offline
Joined: 07/24/2010

I am not sure I understand your problem. You apparently want a separate script to monitor a given file and reset the connection if the file has not not been modifies for some time. Something like that (a script I have not tested):
#!/bin/sh
INTERVAL=5
if [ -n "$2" ]
then
INTERVAL=$2
elif [ -z "$1" -o "$1" = "-h" -o "$1" = "--help" ]
then
printf "Usage: $0 file [interval]
Reset the network connection whenever file has not been modified
during the last interval ($INTERVAL seconds by default). This script should
be killed after file is not supposed to be modified anymore.
"
exit
fi
id="$(nmcli -t -f NAME c show --active)"
while :
do
sleep $INTERVAL
sync "$1"
expr $(date +%s) - $(stat -c %Y "$1") \> $INTERVAL && nmcli connection down "$id" && sleep 5 && nmcli connection up "$id"
done > /dev/null

amenex
Offline
Joined: 01/04/2015

After spending a couple of days experimenting with the overall script in the hope of finding a way of avoiding the stalls
that prevented me from using dig -x to take advantage of its output details without spending several hours at a time
watching for the recurring stalls in the progression of the output of the script, it dawned on me that I might build the
solution into the dig command. The solution:
dig -x +tries=10 100.24.191.100 >> A-16MB.txt ;
dig -x +tries=10 100.24.191.101 >> A-16MB.txt ;
dig -x +tries=10 100.24.191.102 >> A-16MB.txt ;
...
dig -x +tries=10 98.185.162.7 >> A-16MB.txt ;
dig -x +tries=10 98.185.162.8 >> A-16MB.txt ;
dig -x +tries=10 98.185.162.9 >> A-16MB.txt ;
awk '{print $0}' A-16MB.txt | grep -A 16 "ANSWER SECTION" '-' | sed 's/;; ANSWER SECTION://g' | awk 'NF=5 {print $5,$1}' | sed 's/\.0\/24//g'| grep "in-addr.arpa" > Temp-Edit.txt ;
paste -d ' ' <(awk '{print $1}' Temp-Edit.txt) <(awk '{print $2}' Temp-Edit.txt | sed 's/\.in-addr\.arpa\.//g' | sed 's/\./\t/g' | awk '{print $4"."$3"."$2"."$1}') > B-16MB-edit.txt ; rm Temp-Edit.txt ;

The next-to-last line of the overall script is meant to process the entire output all-at-once instead of repeating
the script after every DNS lookup, but it's getting stymied by the ends-of-file that are built into the dig responses.
grep "<<>>" A-16MB.txt
captures only the beginning of every dig response, but does reach every one of the dig responses; however:
grep -A 100 "<<>>" A-16MB.txt
captures what may be ends-of-file within those dig responses, so the ensuing processing
...| grep -A 16 "ANSWER SECTION" '-' | sed 's/;; ANSWER SECTION://g' | awk 'NF=5 {print $5,$1}' | sed 's/\.0\/24//g'| grep "in-addr.arpa" > Temp-Edit.txt<
is getting blocked, leaving Temp-Edit.txt woefully underpopulated.
The good news is that all of the DNS lookups completed without my intervention. The bad news is that there is no such
thing as an end-of-file character ... Is there anything that can be done with sed in this regard ?

Looks as though I'll have to do the processing one-at-a-time and hope that the required extra seven tries in the dig
command will still be enough to keep it going until we can perfect the rattle-the-cage-script.

Upon further investigation, dig -x resolves only the first IPv4 address in the list as though I'm blocked.
With nslookup, progress is still possible ...

Success !
nslookup -retry=10 100.24.191.100 | grep "in-addr.arpa" | awk '{print $4,$1}' | sed 's/\.0\/24//g' >> A-16MB-1015.txt ;
nslookup -retry=10 100.24.191.101 | grep "in-addr.arpa" | awk '{print $4,$1}' | sed 's/\.0\/24//g' >> A-16MB-1015.txt ;
nslookup -retry=10 100.24.191.102 | grep "in-addr.arpa" | awk '{print $4,$1}' | sed 's/\.0\/24//g' >> A-16MB-1015.txt ;
...
nslookup -retry=10 98.185.162.7 | grep "in-addr.arpa" | awk '{print $4,$1}' | sed 's/\.0\/24//g' >> A-16MB-1015.txt ;
nslookup -retry=10 98.185.162.8 | grep "in-addr.arpa" | awk '{print $4,$1}' | sed 's/\.0\/24//g' >> A-16MB-1015.txt ;
nslookup -retry=10 98.185.162.9 | grep "in-addr.arpa" | awk '{print $4,$1}' | sed 's/\.0\/24//g' >> A-16MB-1015.txt ;
grep "in-addr.arpa" A-16MB-1015.txt > Temp-Edit-1015.txt ;
paste -d ' ' <(awk '{print $1}' Temp-Edit-1015.txt) <(awk '{print $2}' Temp-Edit-1015.txt | sed 's/\.in-addr\.arpa\.//g' | sed 's/\./\t/g' | awk '{print $4"."$3"."$2"."$1}') > A-16MB-edit-1015.txt ; rm Temp-Edit-1015.txt ;

While very slow at times when retrying, the script updated at least the output time frequently, finished, and processed
the output file to a make list of hostnames and their IPv4 addresses, 70% as many as the original list of IPv4 addresses.

Magic Banana

I am a member!

Offline
Joined: 07/24/2010

it dawned on me that I might build the solution into the dig command.

So you actually had no issue with your network connection but asked for a precise solution in that direction anyway?!

Your scripts are horrible: do not write the processed data (in that case the IP addresses) in the script!, use loops!, do not execute commands that do nothing but copying the input!, avoid useless temporary files!, stop moving fields around for no reason!, stop changing delimiters and changing them back to what they were!

Here is a reasonable script that takes one IP address per line on the standard input (that you can redirect with "< file", file containing the IP addresses) and does the same as your horrible script, as far as I understand:
#!/bin/sh
while read addr
do
nslookup -retry=10 $addr
done | sed -Ene 's:\.0/24::g' -e 's:(.*)\.(.*)\.(.*)\.(.*)\.in-addr\.arpa.* :\4.\3.\2.\1 :p'

amenex
Offline
Joined: 01/04/2015

The following one-liner:
awk '{print $1}' Addr-List-F-16MB.txt | nslookup -retry=10 | sed -Ene 's:\.0/24::g' -e 's:(.*)\.(.*)\.(.*)\.(.*)\.in-addr\.arpa.* :\4.\3.\2.\1 :p' > F-16MB-edit-1016.txt
ran to completion while I was preparing this response, slow in places while nslookup checked the cage door up
to ten times per address, but without stalling. Incidentally, the script found 750 more hosts than the number
of IPv4 addresses queried. Not all 24,000 servers were up, but nslookup kept track of how many were queried.

Forgive me for not realizing that the dig and nslookup commands retry adjustably until long after starting
this thread.

Thanks for the sed commands; the second one doesn't affect the other field in the nslookup response.

Magic Banana

I am a member!

Offline
Joined: 07/24/2010

The following one-liner

Even better. I should have suspected that nslookup (that I had never used) could itself take a list of IP addresses at input.

One thing though. As I have already written in my previous post, commands that do nothing but copying their input are useless. They only make your script (slightly) less efficient and harder to read. Here, you can write:
$ nslookup -retry=10 < Addr-List-F-16MB.txt | sed -Ene 's:\.0/24::g' -e 's:(.*)\.(.*)\.(.*)\.(.*)\.in-addr\.arpa.* :\4.\3.\2.\1 :p' > F-16MB-edit-1016.txt

Also, repeating what I have told you several times, you should, for reusability, write in an executable file with a meaningful name the command without its input/output (that you will specify whenever you execute it):
#!/bin/sh
# Add here a help message, specifying what the script does and the input/output format
nslookup -retry=10 | sed -Ene 's:\.0/24::g' -e 's:(.*)\.(.*)\.(.*)\.(.*)\.in-addr\.arpa.* :\4.\3.\2.\1 :p'

amenex
Offline
Joined: 01/04/2015

Your elegant condensation of the one liner is understood; but I'll have to modify it, as another group of
malformed in-addr.arpa's came to light:
diff -y --suppress-common-lines --width=180 <(grep -vf Pattern-F-16MB.txt F-16MB-edit-1016.txt | awk '{print $2,$1}' | sort -Vk 2,2) <(grep -v "= " A-16MB-edit-1015.txt | sed 's/\.0\/24//g' | sort -Vk 2,2) > Diff-F-16MB-edit-1016-vs-A-16MB-edit-1015.txt
So Magic Banana's script becomes:
nslookup -retry=10 < Addr-List-F-16MB.txt | sed -Ene 's:\.0/24::g' -e 's/\.128-25//g' -e 's:(.*)\.(.*)\.(.*)\.(.*)\.in-addr\.arpa.* :\4.\3.\2.\1 :p' > F-16MB-edit-1016.txt
The address list needs some adjustment once I weed out the innocent bystanders. The interesting ones will
be the addresses with more than one unrelated name, or unequal addresses that all give the same name or
addresses which are frequently off & on, rendering their names difficult to trace.

Magic Banana's second suggestion is important, as I'll be repeating the DNS lookups, and there shouldn't
be so much risk of accidentally overwriting any previous one. Even a false start obliterates an earlier
one with the same output name.

AttachmentSize
Pattern-F-16MB.txt 17 bytes
F-16MB-edit-1016.txt 1.02 MB
A-16MB-edit-1015.txt 698.23 KB
amenex
Offline
Joined: 01/04/2015

Parsing Magic Banana's script:
expr $(date +%s) - $(stat -c %Y A-16C.txt) \> 1 && nmcli radio wifi off && sleep 0.05s && nmcli radio wifi [on]
"If the current time is greater than the time of the last write to file by more than one second, then turn wifi off;
after a half-second of sleep, turn the wifi back on." That's what it's supposed to do. Maybe we're too impatient. I'll try
thirty seconds.

amenex
Offline
Joined: 01/04/2015

This just came up:
nslookup 216.27.10.252
252.10.27.216.in-addr.arpa canonical name = 252.0/24.10.27.216.in-addr.arpa.
252.0/24.10.27.216.in-addr.arpa name = ankle-aide.soreacocks.com.

Also with dig -x
dig -x 216.27.10.252
...
;; ANSWER SECTION:
252.10.27.216.in-addr.arpa. 1800 IN CNAME 252.0/24.10.27.216.in-addr.arpa.
252.0/24.10.27.216.in-addr.arpa. 7199 IN PTR ankle-aide.soreacocks.com.

None of my scripting was involved, but there are a lot of these; they interfere
with my code for extracting the IPv4's from the canonical names.