Sunday, February 03, 2008

[code] Replay web server log

An excuse to play with Ruby while doing something that we can use at work for scalability and performance tesing.

Input -- Web server request log:
2008-01-01 08:36:01 100.0.0.0 - HTTP 200.0.0.0 80 GET /foo.htm - 200 100 1000 1 HTTP/1.1 Mozilla/4.0
2008-01-01 08:36:05 100.0.0.0 - HTTP 201.0.0.0 80 GET /bar.htm - 200 200 2000 1 HTTP/1.1 Mozilla/4.0


Output -- Result log:
Timestamp|ElapsedTime|RespCode|PageSize|Path
2008-31-01 04:23:47|0.703|200|1000|http://www.acme.com/foo.htm
2008-31-01 04:23:49|0.093|200|2000|http://www.acme.com/bar.htm


Ruby program:
require 'net/http'
require 'uri'

# Wandy 2008 January
# Test utility that reads the log and then play it back by making the
# same http requests as in the log file with the same time delay

# Configuration
Domain = 'http://www.acme.com/'
PlayBackFile = 'PlayLog.txt'
ResultDetailsFile = 'PlayLogResult.txt'
PlayBackSpeed = 2 # Set to 3 for three times as fast. etc

# Init Variables and Result File
tLastTime = nil; nSleepTime = nil
open(ResultDetailsFile, 'w') do |f|
f << "Timestamp|ElapsedTime|RespCode|PageSize|Path\n"
f.flush
end

# Read Page
def getURL(sURL)
thCall = Thread.new {
tStart = Time.now
sPage = Net::HTTP.get_response(URI.parse(sURL))
nDiff = Time.now - tStart

open(ResultDetailsFile, 'a') do |f|
f << "#{tStart.strftime("%Y-%d-%m %H:%M:%S")}|#{nDiff.to_s}|#{sPage.code}|#{sPage.body.size.to_s}|#{sURL}\n"
f.flush
end
}
end

# Main Program
File.read(PlayBackFile).each do | sLine |
sLogLine = sLine[/80 GET .* - .../]

if not sLogLine.nil?
tTime = Time.parse(sLine[0,19])
sPath = sLogLine[8,sLogLine.length-14]

if not tLastTime.nil? then
nSleepTime = (tTime-tLastTime)/PlayBackSpeed
sleep(nSleepTime)
end
getURL(Domain + sPath)
tLastTime = tTime
end
# Wait for all the threads to finish before quitting the main thread
# Though I can't seem to get multiple threads to run in parallel,
# so the use of thread is pointless until I figur out how to fix that
Thread.list.each {|t| if t != Thread.main then t.join end}
end