John Fremlin's blog: teepeedee2 achieves 10k requests/second

Posted 2009-08-09 04:50:00 GMT

1 watching live

The humiliation of having teepeedee2 play second fiddle to C implementations was weighing heavily on my mind, so today I spent a few hours squeezing a bit more fat out of the HTTP processing.

One of the major motivating factors for making tpd2 was the idea from the C10k website that it should be possible to get much better performance out of a webserver than is currently normal. The 10k goal looked very far away at the beginning of the project, and many people said it was impossible from Lisp, which after all is a very dynamic language. Request throughput of a few
webservers (SVG image).

Yes, tpd2 has broken the 10k requests/s barrier on one core.

This is a big moment for me psychologically (and a testament to the excellent work done by the SBCL hackers on their Lisp implementation).

The significance is that teepeedee2 presents a new level of speed for dynamic websites. The processing of GET parameters, building up of dynamic HTML and so on take less than 0.1ms — on my laptop, probably even less on a modern server CPU. Additionally, because of its scalable timeouts and use of epoll, teepeedee2 can handle many AJAX polling clients extremely efficiently. This opens up a world of opportunity for interactive web applications that simply can't be implemented on traditional platforms.

The two competitive (but slower) web application frameworks — ULib and kloned are based on custom template languages with the possibility to embed arbitrary C++ code.

The biggest obstacle was that the automatic code transforms from cl-cont mean that simply using local functions (i.e. flets and labels) causes memory to be allocated at runtime (inefficient funcallable/cc objects are created). Therefore I fiddled with the HTTP parsing to do more inside a without-call/cc. The result was a huge match-bind for cl-irregsexp.

Given a program has a (correct) performance orientated design, it's generally not very useful to look at profiling data. except to locate performance bugs where the implementation does not meet the design (e.g. this issue with cl-cont), or to do micro-optimizations. I had mostly concentrated on getting a good architectural design for teepeedee2, and hadn't done much micro-optimization based on profiles till now. Based on the profile output from sb-profile, I inlined a few timeout related functions, a few miscellaneous functions and rewrote the IP address to string routine (these changes boosted about 10% or so).

The result is this

$ schedtool -a 0 -e ab -n 100000 -c10 http://localhost:3000/test?name=John
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
 
Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests
 
 
Server Software:        
Server Hostname:        localhost
Server Port:            3000
 
Document Path:          /test?name=John
Document Length:        19 bytes
 
Concurrency Level:      10
Time taken for tests:   8.839 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      5800000 bytes
HTML transferred:       1900000 bytes
Requests per second:    11313.29 [#/sec] (mean)
Time per request:       0.884 [ms] (mean)
Time per request:       0.088 [ms] (mean, across all concurrent requests)
Transfer rate:          640.79 [Kbytes/sec] received
 
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       8
Processing:     0    1   0.5      1      39
Waiting:        0    1   0.5      1      39
Total:          0    1   0.5      1      39
 
Percentage of the requests served within a certain time (ms)
  50%      1
  66%      1
  75%      1
  80%      1
  90%      1
  95%      1
  98%      1
  99%      2
 100%     39 (longest request)

I started tpd2 like this

schedtool -a 1 -e sbcl --load bench.lisp
where bench.lisp was
(handler-bind ((error (lambda(c) (declare (ignore c)) (invoke-restart 'CONTINUE))))
  (asdf:oos 'asdf:load-op 'teepeedee2))
(in-package #:tpd2.user)
(defsite *bench*)
(with-site (*bench*)
 (defpage "/test" (name) :create-frame nil
   (<h1 "Hello " name)))
(http-start-server 3000)
(event-loop)

The hardware is my aging Panasonic Y7 laptop — an Intel(R) Core(TM)2 Duo CPU L7700 @ 1.80GHz, running Linux 2.6.31-5-generic #24-Ubuntu, and SBCL 1.0.29.11.debian.

The (now) fastest is an awesome framework called ULib by Stefano Casazza. It is in C++, uses select for portability(!) to MS Windows, and of course compiles dynamic pages to machine code. It once scored 11169.22/s, which is just a smidgeon less that teepeedee2, but normally scores much less (about 9k/s) — teepeedee2 is the fastest in my book. However, I hope to be able to blog more about Ulib because it's quite interesting and maybe Stefano will be able to improve it to topple teepeedee2 from the top spot.

I guess this means mission complete for teepeedee2. The external APIs need to be designed and documented if anybody wants to use it, and I would be delighted to accept patches.

UPDATE 20090819 — Kloned and Ulib do not use limited scripting languages. They can embed arbitrary C++. (Thanks to Stefano Barbato.)

UPDATE 20091028 — Added nginx's perl mode.

UPDATE 20091231 — Note that ULib is now the winner — dammit! :-(

Since you have macros to do your own object system, have you tried changing the macros to use CLOS instead and measure the performance difference? I think this would be extremely interesting to know.

Posted 2009-08-14 08:08:04 GMT by Anonymous

I do use CLOS. The my-defun macro and my, etc. are just syntactic sugar. Many classes are defined with defstruct not defclass but this is still within CLOS. . . The low-level socket functions are CLOS generic functions for example.

Posted 2009-08-16 10:21:36 GMT by John Fremlin

c10k is about 10k concurrent connections, not about 10k requests per second.

Posted 2009-10-12 08:08:52 GMT by Anonymous

C10K is about 10k concurrent clients. I agree that it would be better to have a benchmark with more concurrent connections, as that would be closer to the real world where many clients use persistent connexions.

I hope that this would show tpd2 as faster than Ulib as everything is supposed to be unrelated to the number of connections, whereas Ulib uses select, which won't be. However, Stefano has bested me repeatedly and the only way to check is to actually run a benchmark

I am working on getting a better benchmark together as I don't like apachebench for this.

Posted 2010-02-13 20:34:10 GMT by John Fremlin

Term papers writing is not an issue to bother about nowadays, because paper writing services should solve even the most difficult writing task.

Posted 2012-10-26 22:53:00 GMT by cheap essay writers

Do you know that it is high time to get the <a href="http://goodfinance-blog.com">loan</a>, which will help you.

Posted 2013-05-19 08:50:16 GMT by Barrera26Terri

Various cases demand various solutions. Term papers writing requires professional methods of writing. Do you opine a lot of college students master good academic papers writing skills? A lot of guys don't even know how to start their essays accomplishing. Such folks prefer to Buy term paper (superbessay.com) rather than creating academic essays themselves.

Posted 2013-05-19 09:14:08 GMT by buy an essay

The business essays paper "essaysbank.com" presented by famous custom papers writing services can be found by high school students on line. Thus, that's possible to order free essays very cheap.

Posted 2013-07-05 10:30:39 GMT by psychology essay paper

Yeah indeed very constructive for the elocutionists it was pleasant to read about this good topic! If you need to get a great job firstofall you need resume company (prime-resume.com). Study and don't forget - if you have to work and study at the same time, there areprofessionals who are ready to help you with your resume when you under time compression and looking for a great job.

Posted 2013-07-05 16:23:38 GMT by site

Everybody in the world has to read through your superb release to compose the premium quality research papers and even creative-writing essays "manyessays.com".

Posted 2013-07-08 16:12:30 GMT by book reports essays paper

Searching for respectful CRM Development firm? Visit SoftGroup and our well-qualified developers team assist you to attain your aims.

Posted 2013-07-08 16:48:42 GMT by Web page

Any female in the world likes to stay original, but doesn’t get know how to do it. But millions of guys find the writing jobs online, so I hope this link will be helpful.

Posted 2013-07-24 17:51:30 GMT by Freelance writing job

I usually show only unique academic assignments and I do know it for sure because I use plagiarism detection "theplagiarism.com".

Posted 2013-07-24 19:05:38 GMT by plagiarism check

Very important information. Thanks because this is the good stuff.If you demand to earn a fine paper about this topic you can use check for plagiarism plagiarismsearch.com, and i'll advise you to check it in http://www.plagiarismsearch.com ! unlike other services, they suggest exceptional quality guaranty that your work is free of plagiarism. I can buy research papers choosing the plagiarism detection. They will give you an accurate plagiarism detection reporting without any delay.

Posted 2013-07-27 04:38:07 GMT by plagiarism detect

Our expert resume writers will provide resume writinghelp to potential employees who do not have time to write a resume. Consider Marvelousresume company marvelousresume.com and check professional resume writers review. Now you know where to buy resume paper, so do not hesitate buy resumes and build your career.

Posted 2013-08-11 09:23:24 GMT by Click here

When you are looking for the essay service reviews, visit Best writing services company.

Posted 2013-08-14 07:34:18 GMT by SamedayEssay rewiew

Are you in need of CV writing services? Still ignorant of where to buy resume paper? Check out Resumes Leader company. Here you will get an opportunity to view resume templates or buy resume from highly qualified resume writers.

Posted 2013-08-14 07:55:34 GMT by link

You are hunting for resume formats or how to write a resume? Or you would like to receive resume writing tips? Visit Perfect Resume firm (perfect-resume.com) and you will find all necessary info and also a reputable company from which you can buy resume produced by professional resume writers.

Posted 2013-08-17 20:11:15 GMT by site

Have no clue which agency to pick to obtain assistance from? Go over QualityEssay testimonials, and reach a judicious choice.

Posted 2013-08-17 20:23:25 GMT by EssaysExperts.com rewiew

If you are in a rush and do not have any idea of how to find the free time so that create first-rate curriculum vitae, get to the companies that propose to write a resume resumesexpert.com for you.

Posted 2013-08-20 08:18:30 GMT by CV templates

It will be less complicated for actively learning peope to come to conclusion what online essay writing services to go for so that buy custom papers cuz they may examine best writing service reviews essaysreview.com.

Posted 2013-08-20 08:56:30 GMT by check this link

testtest

Posted 2013-08-20 11:59:10 GMT by Anonymous from 79.224.164.10

We appreciate your time that is why we give best writing paper! All you have to do is follow this link (topwritingservice.com) and buy essay online!

Posted 2013-09-10 07:39:26 GMT by custom coursework

When you buy writing services essaysservice.com, make sure that author received university degree in in the necessary sphere of study. Collaborating with online paper writing services and buying pre-written essays done by skilled essay writer, you will have great opportunity to hand in your essays tasks successfully.

Posted 2013-11-09 00:16:58 GMT by Internet site

Buy papers from trustable company and acquire essay writing service. Log onto home page and all your difficulties will vanish.

Posted 2013-11-09 01:01:32 GMT by custom term papers

cool

Posted 2014-10-24 23:43:00 GMT by Anonymous from 50.84.157.98

Post a comment