Sep 24

PALO ALTO, Calif.–Facebook has unleashed a Tornado, and it’s hoping that some eager engineers will go catch it.

Earlier this month, Facebook released the open-source Web server framework called Tornado, which powers the real-time streaming behind its latest toy, social feed aggregator FriendFeed. And on Wednesday evening at the office that most recently housed the FBFund incubator program, senior open programs manager David Recordon and director of products Bret Taylor held a "tech talk" to pitch Tornado to a crowd of several dozen interested members of the Web development community.

"We had actually been planning on open-sourcing (Tornado)" prior to Facebook’s acquisition of FriendFeed, said Taylor, who had served as CEO of the start-up. "When we got to Facebook we thought it was a really good opportunity to do it."

The slant of Wednesday evening’s talk (which was quite technical, so I won’t be going into significant detail): if you’re dealing with real-time, streaming content, Facebook thinks Tornado is for you. And if you’ve been listening to anything that Facebook has been saying recently, it believes the real-time Web is the future for everyone–not just its own company.

"FriendFeed’s a real-time system," Taylor said as he described how the Python-based Tornado framework’s non-blocking nature was ideal for real-time Web services. "Essentially, every active user of FriendFeed maintains an open connection to the FriendFeed servers."

Both Recordon and Taylor are recent arrivals at Facebook: Recordon joined Facebook last month as its resident open-source guru, and the company had acquired FriendFeed a few weeks earlier in a deal that brought on board both a top-notch engineering team (its founders, including Taylor, were Google veterans) and cutting-edge technology for amassing and indexing real-time Web conversations–so cutting-edge, in fact, that it was unclear as to how the mainstream would ever actually accept it.

At the time, there were questions about what, exactly, Facebook would actually do with FriendFeed. In the meantime it’s become clear that acquiring the would-be Twitter rival allowed Facebook to leap ahead with some of its development of new, real-time-focused features as well as to enhance existing ones with FriendFeed’s technology and brainpower.

Open-sourcing the technology doesn’t have an obvious financial end for Facebook. But it will ideally mean that some of the developer community will be marching to Facebook’s beat, at a time when the company continues to compete with the far smaller Twitter for a majority share of what’s come to be known as the real-time Web.

As for its Python foundations, Taylor said that FriendFeed had been looking to build Tornado in a manner "sophisticated enough that we could do all the things we wanted but well known enough so that a new engineer could theoretically understand our code base right away…Python has a lot of its flaws, I wish it had real inline functions like Javascript, but for all of its flaws it’s actually pretty nice to use in practice."

Taylor told me afterward that no concrete plans have been put into action as to which Facebook features may be getting a FriendFeed makeover (so as to speak) but hinted that one getting talked about for some enhancement from the former FriendFeed team is Facebook Chat, the site’s instant messaging client, because of its obviously real-time nature.

Tornado isn’t the first technology that Facebook, still criticized by some of the open-source community for its heavy reliance on proprietary technology and a login wall, has released as open-source code: well over a year ago, the company released the code for a significant portion of its developer platform.

Tagged with:
May 25

Nikto is an Open Source (GPL) web server scanner which performs comprehensive tests against web servers for multiple items, including over 3500 potentially dangerous files/CGIs, versions on over 900 servers, and version specific problems on over 250 servers. Scan items and plugins are frequently updated and can be automatically updated (if desired).

Nikto is not designed as an overly stealthy tool. It will test a web server in the shortest timespan possible, and it’s fairly obvious in log files. However, there is support for LibWhisker’s anti-IDS methods in case you want to give it a try (or test your IDS system).

Not every check is a security problem, though most are. There are some items that are "info only" type checks that look for items that may not have a security flaw, but the webmaster or security engineer may not know are present on the server. These items are usually marked appropriately in the information printed. There are also some checks for unknown items which have been seen scanned for in log files.

Nikto Site:http://www.cirt.net/nikto2

Download nikto:
wget http://www.cirt.net/nikto/nikto-current.tar.gz

Install:
tar zxvf  nikto-current.tar.gz

Nikto Help:
[root@localhost nikto]# ./nikto.pl -h
Option host requires an argument

       -Cgidirs+                scan these CGI dirs: ‘none’, ‘all’, or values like "/cgi/ /cgi-a/"
       -dbcheck                 check database and other key files for syntax errors (cannot be abbreviated)
       -evasion+                ids evasion technique
       -Format+                 save file (-o) format
       -host+                   target host
       -Help                    Extended help information
       -id+                     host authentication to use, format is userid:password
       -mutate+                 Guess additional file names
       -output+                 write output to this file
       -port+                   port to use (default 80)
       -Display+                turn on/off display outputs
       -ssl                     force ssl mode on port
       -Single                  Single request mode
       -timeout+                timeout (default 2 seconds)
       -Tuning+                 scan tuning
       -update                  update databases and plugins from cirt.net (cannot be abbreviated)
       -Version                 print plugin and database versions
       -vhost+                  virtual host (for Host header)
   + requires a value

Example:

1.Basic Testing

The most basic Nikto scan requires simply a host to target, since port 80 is assumed if none is specified. The host can either be an IP or a hostname of a machine, and is specified using the -h (-host) option. This will scan the IP 192.168.0.1 on TCP port 80:

perl nikto.pl -h 192.168.1.10

To check on a different port, specify the port number with the -p (-port) option. This will scan the IP 192.168.0.1 on TCP port 443:

perl nikto.pl -h 192.168.1.10 -p 443

Hosts, ports and protocols may also be specified by using a full URL syntax, and it will be scanned:

perl nikto.pl -h https://192.168.1.10:443/

There is no need to specify that port 443 may be SSL, as Nikto will first test regular HTTP and if that fails, HTTPS. If you are sure it is an SSL server, specifying -s (-ssl) will speed up the test.

perl nikto.pl -h 192.168.1.10 -p 443 –ssl

2. Multiple Port Testing

Nikto can scan multiple ports in the same scanning session. To test more than one port on the same host, specify the list of ports in the -p (-port) option. Ports can be specified as a range (i.e., 80-90), or as a comma-delimited list, (i.e., 80,88,90). This will scan the host on ports 80, 88 and 443.

perl nikto.pl -h 192.168.1.10 -p 80,88,443

3. Multiple Host Testing

Nikto support scanning multiple hosts in the same session via a text file of host names or IPs. Instead of giving a host name or IP for the -h (-host) option, a file name can be given. A file of hosts must be formatted as one host per line, with the port number(s) at the end of each line. Ports can be separated from the host and other ports via a colon or a comma. If no port is specified, port 80 is assumed.

This is an example of a valid hosts file:
192.168.1.1:80
192.168.1.2,80
192.168.1.3
192.168.1.10,80,443
192.168.1.10:80:443
localhost:8888

4. Using a Proxy

If the machine running Nikto only has access to the target host (or update server) via an HTTP proxy, the test can still be performed. Set the PROXY* variables (as described in section 4), then execute Nikto with the -u (-useproxy) command. All connections will be relayed through the HTTP proxy specified in the configuration file.

perl nikto.pl -h 192.168.1.10 -p 80 –u

5. Updating

Nikto can be automatically updated, assuming you have Internet connectivity from the host Nikto is installed on. To update to the latest plugins and databases, simply run Nikto with the -update command.

perl nikto.pl -update

Note:

The -update option cannot be abbreviated.

If updates are required, you will see a list of the files downloaded:

perl nikto.pl –update
+ Retrieving ‘nikto_core.plugin’
+ Retrieving ‘CHANGES.txt’

Updates may also be manually downloaded from http://www.cirt.net/

Tagged with:
Jan 06

Web Bench is very simple tool for benchmarking WWW or proxy servers. Uses fork() for simulating multiple clients and can use HTTP/0.9-HTTP/1.1 requests. This benchmark is not very realistic, but it can test if your HTTPD can realy handle that many clients at once (try to run some CGIs) without taking your machine down. Displays pages/min and bytes/sec. Can be used in more aggressive mode with -f switch.

INSTALL:

wget http://www.goitworld.com/download/webbench-1.5.tar.gz
tar -zxvf webbench-1.5.tar.gz
cd webbench-1.5
make && make install

Options:

webbench [option]… URL
  -f|–force               Don’t wait for reply from server.
  -r|–reload              Send reload request – Pragma: no-cache.
  -t|–time <sec>          Run benchmark for <sec> seconds. Default 30.
  -p|–proxy <server:port> Use proxy server for request.
  -c|–clients <n>         Run <n> HTTP clients at once. Default one.
  -9|–http09              Use HTTP/0.9 style requests.
  -1|–http10              Use HTTP/1.0 protocol.
  -2|–http11              Use HTTP/1.1 protocol.
  –get                    Use GET request method.
  –head                   Use HEAD request method.
  –options                Use OPTIONS request method.
  –trace                  Use TRACE request method.
  -?|-h|–help             This information.
  -V|–version             Display program version.

Test Result:

#webbench -c 500 -t 30 http://localhost

Webbench – Simple Web Benchmark 1.5
Copyright (c) Radim Kolar 1997-2004, GPL Open Source Software.

Benchmarking: GET http://localhost/
500 clients, running 30 sec.

Speed=350 pages/min, 356755 bytes/sec.
Requests: 175 susceed, 0 failed.

Tagged with:
Jan 06

ab is a tool for benchmarking the performance of your WWW HyperText Transfer Protocol (HTTP) server. It does this by giving you an indication of how many requests per second your Apache installation can serve.

Options

-A auth-username:password
Supply BASIC Authentication credentials to the server. The username and password are separated by a single : and sent on the wire base64 encoded. The string is sent regardless of whether the server needs it (i.e., has sent an 401 authentication needed).

-b windowsize
Size of TCP send/receive buffer, in bytes.

-c concurrency
Number of multiple requests to perform at a time. Default is one request at a time.

-C cookie-name=value
Add a Cookie: line to the request. The argument is typically in the form of a name=value pair. This field is repeatable.

-d
Do not display the "percentage served within XX [ms] table". (legacy support).

-e csv-file
Write a Comma separated value (CSV) file which contains for each percentage (from 1% to 100%) the time (in milliseconds) it took to serve that percentage of the requests. This is usually more useful than the ‘gnuplot’ file; as the results are already ‘binned’.

-f protocol
Specify SSL/TLS protocol (SSL2, SSL3, TLS1, or ALL).

-g gnuplot-file
Write all measured values out as a ‘gnuplot’ or TSV (Tab separate values) file. This file can easily be imported into packages like Gnuplot, IDL, Mathematica, Igor or even Excel. The labels are on the first line of the file.

-h
Display usage information.

-H custom-header
Append extra headers to the request. The argument is typically in the form of a valid header line, containing a colon-separated field-value pair (i.e., "Accept-Enooding: zip/zop;8bit").

-i
Do head requests instead of GET.

-k
Enable the HTTP KeepAlive feature, i.e., perform multiple requests within one HTTP session. Default is no KeepAlive.

-n requests
Number of requests to perform for the benchmarking session. The default is to just perform a single request which usually leads to non-representative benchmarking results.

-p POST-file
File containing data to POST. Remember to also set -T.

-P proxy-auth-username:password
Supply BASIC Authentication credentials to a proxy en-route. The username and password are separated by a single : and sent on the wire base64 encoded. The string is sent regardless of whether the proxy needs it (i.e., has sent an 407 proxy authentication needed).

-q
When processing more than 150 requests, ab outputs a progress count on stderr every 10% or 100 requests or so. The -q flag will suppress these messages.

-r
Don’t exit on socket receive errors.

-s
When compiled in (ab -h will show you) use the SSL protected https rather than the http protocol. This feature is experimental and very rudimentary. You probably do not want to use it.

-S
Do not display the median and standard deviation values, nor display the warning/error messages when the average and median are more than one or two times the standard deviation apart. And default to the min/avg/max values. (legacy support).

-t timelimit
Maximum number of seconds to spend for benchmarking. This implies a -n 50000 internally. Use this to benchmark the server within a fixed total amount of time. Per default there is no timelimit.

-T content-type
Content-type header to use for POST data, eg. application/x-www-form-urlencoded. Default: text/plain.

-v verbosity
Set verbosity level – 4 and above prints information on headers, 3 and above prints response codes (404, 200, etc.), 2 and above prints warnings and info.

-V
Display version number and exit.

-w
Print out results in HTML tables. Default table is two columns wide, with a white background.

-x <table>-attributes
String to use as attributes for <table>. Attributes are inserted <table here >.

-X proxy[:port]
Use a proxy server for the requests.

-y <tr>-attributes
String to use as attributes for <tr>.

-z <td>-attributes
String to use as attributes for <td>.

-Z ciphersuite
Specify SSL/TLS cipher suite (See openssl ciphers).

Test Result

#ab -n 20 -c 10 -t 60 http://www.localhost.com/

Benchmarking www.localhost.com (be patient)
Finished 154 requests

Server Software:        Apache/2.2.3
Server Hostname:       
www.localhost.com
Server Port:            80

Document Path:          /
Document Length:        56611 bytes

Concurrency Level:      10
Time taken for tests:   60.5793 seconds
Complete requests:      154
Failed requests:        0
Write errors:           0
Total transferred:      8933962 bytes
HTML transferred:       8906422 bytes
Requests per second:    2.57 [#/sec] (mean)
Time per request:       3896.480 [ms] (mean)
Time per request:       389.648 [ms] (mean, across all concurrent requests)
Transfer rate:          145.39 [Kbytes/sec] received

Connection Times (ms)
                       min  mean[+/-sd] median   max
Connect:      257  610 1192.0    258    9256
Processing:  1037 2848 2052.2   2076   13469
Waiting:      258  454 872.4    259   10038
Total:       1296 3459 2368.4   2591   14335

Percentage of the requests served within a certain time (ms)
  50%   2591
  66%   3892
  75%   4672
  80%   5163
  90%   6588
  95%   7761
  98%   9361
  99%  13729
100%  14335 (longest request)

Refrence:       http://httpd.apache.org/
Tagged with:
preload preload preload