I've been in need of a tool that would take two different sets of port numbers, and subtract one set from the other. This would be similar to comm -13, but would be able to act upon ranges in a list (e.g. 8,12,23-56).

Scenario

Here's a scenario. You've been scanning a set of computers for the top 10 nmap ports: 21-23,25,80,110,139,443,445,3389. You want to move on to the top 20: 21-23,25,53,80,110-111,135,139,143,443,445,993,995,1723,3306,3389,5900,8080. There's no point in re-scanning the top 10, you can cut your time in half by only scanning the new ports (top 10-20, not 1-20). I wanted something that could take two lists in this format, and give me the new ports that need to be scanned.

The sample size here is small. Imagine though that you've been scanning the Nmap top 1000 ports, and you want to find out what other odd IANA ports might be opened, that's 6140 ports! You can reduce the number of ports you have to scan by ~1/6th by removing the Nmap top 1000 from that list. Place the top 1000 ports in small.txt, and the IANA TCP ports in the large.txt.

Compress individual results

One more thing this is handy for. I found that masscan will truncate a list if it's too large. At one point I fed it a .conf file with a very long list of individual port numbers. Rather than scanning about 40,000 ports, it was scanning ~22,000. Using --echo, I discovered it was truncating the line containing my port numbers. Woops. Feed that long list of individual ports into long.txt, and put the number 0 into short.txt, and it will diff out the 0 and return the results in a much shorter (character length wise) format with ranges. In other words, if you feed it 80,81,82,83, it will return 80-83.

The code:

#!/usr/bin/python3

import itertools

shortFile = "short.txt"
longFile = "long.txt"
shortList = []
longList = []
diffedList = []

# -- Flow --
# Take the two lists, expand both of them
# subtract the small list from the large list
# contract the results
# Format and output results

# Our Short List
with open(shortFile) as f:
	lines = f.readlines()
	for line in lines:
		tempList = line.split(",")
	for i in range(len(tempList)):
		item = ''.join(tempList[i])
		if '-' in item: # If the item is a range, expand it
			first,second = item.split("-")
			b = range(int(first),int(second)+1)
			c = [*b]
			for i in range(len(c)):
				shortList.append(c[i])
		else:
			shortList.append(int(item))

 # Our Long List
with open(longFile) as f:
	lines = f.readlines()
	for line in lines:
		tempList = line.split(",")
	for i in range(len(tempList)):
		item = ''.join(tempList[i])
		if '-' in item: # If the item is a range, expand it
			first,second = item.split("-")
			b = range(int(first),int(second)+1)
			c = [*b]
			for i in range(len(c)):
				longList.append(c[i])
		else:
			longList.append(int(item))

# Get the diff of the lists
diffedList = [x for x in longList if x not in shortList]

# Output the number of items in the list, just for information.
print('\n',end='')
print(str(len(sorted(set(diffedList)))) + " Ports") 
print('\n',end='')

# turns individual ports into port ranges: list(1,2,3) into range(1, 3)
def ranges(p):
	q = sorted(set(p))
	i = 0
	for j in range(1,len(q)):
		if q[j] > 1+q[j-1]:
			yield (q[i],q[j-1])
			i = j
	yield (q[i], q[-1])

diffedList = list(ranges(diffedList))

# For each item, if it's a range of 0 (e.g. range(23, 23), just print that value)
# For each other value, change the format from (56, 74) to 56-74
# and print that for use.
for i in range(len(diffedList)):
	if len(range(diffedList[i][0],diffedList[i][1])) == 0:
		print(diffedList[i][0],end='')
	else:
		item = str(diffedList[i])
		item = item.replace(", ", "-")
		item = item.replace("(", "")
		item = item.replace(")", "")
		print(item,end='')
	if i <= len(diffedList)-2:
		print(",",end='')
print('\n')