Skip to content
GitLab

RPC API · Changes

Page history
RPC-API.textile => RPC-API.md authored by Tasos Laskos's avatar Tasos Laskos
Show whitespace changes
Inline Side-by-side
guides/developer/RPC-API.md 0 → 100644
View page @ 6626ed3f
Version 0.4.1
-------------
First of all, you should take a look at the
<a href="https://github.com/Arachni/arachni-rpc">Arachni-RPC</a>
protocol itself especially its
<a href="https://github.com/Arachni/arachni-rpc/wiki">design specification</a>.
To provide functional examples of RPC interaction, I"ll be using the
<a href="https://github.com/Arachni/arachni-rpc-pure">pure Ruby client
implementation</a> of the Arachni-RPC protocol.
If you are interested in the full API you can browse through it
[here](http://arachni.github.com/arachni/Arachni/RPC/Server.html) -- the
accessible classes are Dispatcher, Instance and Framework.
In order to perform an audit you have to go through the following
straightforward steps:
1. [Connect to an Arachni RPC Dispatch server](#dispatcher_connect)
2. [Request an Arachni instance](#dispatch)
3. [Retrieve framework components (Optional)](#retrieve)
1. [Modules](#retrieve_modules)
2. [Plugins](#retrieve_plugins)
4. [Set the options you want](#options)
1. [Setting the URL (opts.url=)](#options_url)
2. [Audit links (opts.audit_links=)](#options_audit_links)
3. [Audit forms (opts.audit_forms=)](#options_audit_forms)
4. [Audit cookies (opts.audit_cookies=)](#options_audit_cookies)
5. [Audit headers (opts.audit_headers=)](#options_audit_headers)
6. [Link count limit (opts.link-count-limit=)](#options_link_count_limit)
7. [Exclude cookies (opts.exclude_cookies=)](#options_exclude_cookies)
8. [Exclude vectors (opts.exclude_vectors=)](#options_exclude_vectors)
9. [User agent (opts.user_agent=)](#options_user_agent)
10. [Exclude patterns (opts.exclude=)](#options_exclude)
11. [Include patterns (opts.include=)](#options_include)
12. [Cookie jar (opts.cookie_jar=)](#options_cookie_jar)
13. [HTTP request limit (opts.http-req-limit=)](#options_http_req_limit)
14. [Redundancy rules (opts.redundant=)](#options_redundant)
5. [Load the modules you want to run](#modules)
6. [Load the plugins you want to run](#plugins)
7. [Run the framework](#run)
1. [Get the progress while busy](#run_busy)
2. [Pausing and Resuming](#run_pause_resume)
8. [Get the report](#report)
9. [Shutdown the server](#shutdown)
10. [Cancelling the scan](#cancel)
### <a id="example" href="#example">A minimalistic example</a>
Setting up the test environment
-------------------------------
First of all, we"ll need to run an Arachni RPC Dispatcher to have
something to work and play with.
$ arachni_rpcd
Arachni - Web Application Security Scanner Framework v0.4.1dev
Author: Tasos "Zapotek" Laskos <tasos.laskos@gmail.com>
(With the support of the community and the Arachni Team.)
Website: http://arachni-scanner.com
Documentation: http://arachni-scanner.com/wiki
Arachni - Web Application Security Scanner Framework v0.4.1dev
Author: Tasos "Zapotek" Laskos <tasos.laskos@gmail.com>
(With the support of the community and the Arachni Team.)
Website: http://arachni-scanner.com
Documentation: http://arachni-scanner.com/wiki
I, [2012-08-28T05:29:39.412457 #23997] INFO -- System: RPC Server started.
I, [2012-08-28T05:29:39.412557 #23997] INFO -- System: Listening on localhost:1605
Arachni - Web Application Security Scanner Framework v0.4.1dev
Author: Tasos "Zapotek" Laskos <tasos.laskos@gmail.com>
(With the support of the community and the Arachni Team.)
Website: http://arachni-scanner.com
Documentation: http://arachni-scanner.com/wiki
I, [2012-08-28T05:29:39.419214 #24001] INFO -- System: RPC Server started.
I, [2012-08-28T05:29:39.419314 #24001] INFO -- System: Listening on localhost:54160
[...lots of similar output...]
This is what happens when no options have been set; the default port is _7331_.
<a id="dispatcher_connect" href="#dispatcher_connect">Connect to an Arachni RPC Dispatch server</a>
---------------------------------------------------------------
First of all, install the Arachni-RPC Pure client:
gem install arachni-rpc-pure
Simple as:
```ruby
require 'arachni/rpc/pure'
dispatcher = Arachni::RPC::Pure::Client.new(
host: 'localhost',
port: 7331
)
```
<a id="dispatch" href="#dispatch">Request an Arachni instance</a>
----------------------------------------
```ruby
# request for an instance to be dispatched
instance_info = dispatcher.call( 'dispatcher.dispatch' )
```
<a id="instance_connect" href="#instance_connect">Connect to the Arachni RPC instance</a>
--------------------------------------------------------
Simple as:
```ruby
host, port = instance_info['url'].split( ':' )
instance = Arachni::RPC::Pure::Client.new(
host: host,
port: port,
token: instance_info['token']
)
```
**In order to successfully authenticate yourself to the instance don't forget
to include the authentication token.**
<a id="retrieve" href="#retrieve">Retrieve framework components (Optional)</a>
-----------------------------------------------------
This is strictly optional and is only useful when you"re developing an interface
and want to show the user all available components -- or in similar situations.
### <a id="retrieve_modules" href="#retrieve_modules">Modules (framework.lsmod)</a>
To retrieve all available modules:
```ruby
instance.call( 'framework.lsmod' )
```
Which will return something like:
[
[ 0] {
:name => "Code injection",
:description => "It tries to inject code snippets into the\n web application and assess whether or not the injection\n was successful.",
:elements => [
[0] "form",
[1] "link",
[2] "cookie",
[3] "header"
],
:author => [
[0] "Tasos \"Zapotek\" Laskos <tasos.laskos@gmail.com>"
],
:version => "0.1.6",
:references => {
"PHP" => "http://php.net/manual/en/function.eval.php",
"Perl" => "http://perldoc.perl.org/functions/eval.html",
"Python" => "http://docs.python.org/py3k/library/functions.html#eval",
"ASP" => "http://www.aspdev.org/asp/asp-eval-execute/",
"Ruby" => "http://en.wikipedia.org/wiki/Eval#Ruby"
},
:targets => [
[0] "PHP",
[1] "Perl",
[2] "Python",
[3] "ASP",
[4] "Ruby"
],
:issue => {
:name => "Code injection",
:description => "Arbitrary code can be injected into the web application\n which is then executed as part of the system.",
:tags => [
[0] "code",
[1] "injection",
[2] "regexp"
],
:cwe => "94",
:severity => "High",
:cvssv2 => "7.5",
:remedy_guidance => "User inputs must be validated and filtered\n before being evaluated as executable code.\n Better yet, the web application should stop evaluating user\n inputs as any part of dynamic code altogether.",
:remedy_code => "",
:metasploitable => "unix/webapp/arachni_php_eval"
},
:mod_name => "code_injection",
:path => "/home/zapotek/workspace/arachni/modules/audit/code_injection.rb"
},
[ 1] {
:name => "PathTraversal",
:description => "It injects paths of common files (/etc/passwd and boot.ini)\n and evaluates the existence of a path traversal vulnerability\n based on the presence of relevant content in the HTML responses.",
:elements => [
[0] "form",
[1] "link",
[2] "cookie",
[3] "header"
],
:author => [
[0] "Tasos \"Zapotek\" Laskos <tasos.laskos@gmail.com>"
],
:version => "0.2.6",
:references => {
"OWASP" => "http://www.owasp.org/index.php/Path_Traversal",
"WASC" => "http://projects.webappsec.org/Path-Traversal"
},
:targets => [
[0] "Unix",
[1] "Windows",
[2] "Tomcat"
],
:issue => {
:name => "Path Traversal",
:description => "The web application enforces improper limitation\n of a pathname to a restricted directory.",
:tags => [
[0] "path",
[1] "traversal",
[2] "injection",
[3] "regexp"
],
:cwe => "22",
:severity => "Medium",
:cvssv2 => "4.3",
:remedy_guidance => "User inputs must be validated and filtered\n before being used as a part of a filesystem path.",
:remedy_code => "",
:metasploitable => "unix/webapp/arachni_path_traversal"
},
:mod_name => "path_traversal",
:path => "/home/zapotek/workspace/arachni/modules/audit/path_traversal.rb"
},
[...and many more...]
]
### <a id="retrieve_plugins" href="#retrieve_plugins">Plugins (framework.lsplug)</a>
To retrieve all available plugins
```ruby
instance.call( 'framework.lsplug' )
```
[
[ 0] {
:name => "Resolver",
:description => "Resolves vulnerable hostnames to IP addresses.",
:author => [
[0] "Tasos \"Zapotek\" Laskos <tasos.laskos@gmail.com>"
],
:tags => [
[0] "ip address",
[1] "hostname"
],
:version => "0.1.1",
:plug_name => "resolver",
:path => "/home/zapotek/workspace/arachni/plugins/defaults/resolver.rb",
:options => []
},
[ 1] {
:name => "Health map",
:description => "Generates a simple list of safe/unsafe URLs.",
:author => [
[0] "Tasos \"Zapotek\" Laskos <tasos.laskos@gmail.com>"
],
:version => "0.1.3",
:plug_name => "healthmap",
:path => "/home/zapotek/workspace/arachni/plugins/defaults/healthmap.rb",
:options => []
},
[...and many more...]
]
<a id="options" href="#options">Set the options you want</a>
------------------------------------
You can do that in 2 ways, either set them all at once or one at a time.
To set them all at once:
```ruby
# you can also use Strings as keys
opts = {
url: 'http://demo.testfire.net',
audit_links: true,
audit_forms: true,
audit_cookies: true,
audit_headers: true,
link_count_limit: 1,
user_agent: "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; en-us) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/5.0.3Safari/5",
}
p instance.call( 'opts.set', opts )
#=> true
```
You can use any of the available options that will be presented later in the ```opts``` hash.
### <a id="options_url" href="#options_url">Setting the URL (opts.url=)</a>
**Expects**: String <br/>
**Default**: <n/a>
To set the URL of the site you want to audit:
```ruby
p instance.call( 'opts.url=', 'http://demo.testfire.net' )
#=> "http://demo.testfire.net"
```
### <a id="options_audit_links" href="#options_audit_links">Audit links (opts.audit_links=)</a>
**Expects**: Boolean <br/>
**Default**: false
To audit link elements:
```ruby
p instance.call( 'opts.audit_links=', true )
#=> true
p instance.call( 'opts.audit', :links )
#=> true
p instance.call( 'opts.audit?', :links )
#=> true
p instance.call( 'opts.dont_audit', :links )
#=> true
p instance.call( 'opts.audit?', :links )
#=> false
```
### <a id="options_audit_forms" href="#options_audit_forms">Audit forms (opts.audit_forms=)</a>
**Expects**: Boolean <br/>
**Default**: false
To audit form elements:
```ruby
p instance.call( 'opts.audit_forms=', true )
#=> true
p instance.call( 'opts.audit', :forms )
#=> true
p instance.call( 'opts.audit?', :forms )
#=> true
p instance.call( 'opts.dont_audit', :forms )
#=> true
p instance.call( 'opts.audit?', :forms )
#=> false
```
### <a id="options_audit_cookies" href="#options_audit_cookies">Audit cookies (opts.audit_cookies=)</a>
**Expects**: Boolean <br/>
**Default**: false
To audit cookies:
```ruby
p instance.call( 'opts.audit_cookies=', true )
#=> true
p instance.call( 'opts.audit', :cookies )
#=> true
p instance.call( 'opts.audit?', :cookies )
#=> true
p instance.call( 'opts.dont_audit', :cookies )
#=> true
p instance.call( 'opts.audit?', :cookies )
#=> false
```
### <a id="options_audit_headers" href="#options_audit_headers">Audit headers (opts.audit_headers=)</a>
**Expects**: Boolean <br/>
**Default**: false
To audit headers:
```ruby
p instance.call( 'opts.audit_headers=', true )
#=> true
p instance.call( 'opts.audit', :headers )
#=> true
p instance.call( 'opts.audit?', :headers )
#=> true
p instance.call( 'opts.dont_audit', :headers )
#=> true
p instance.call( 'opts.audit?', :headers )
#=> false
```
### <a id="options_link_count_limit" href="#options_link_count_limit">Link count limit (opts.link_count_limit=)</a>
**Expects**: Integer <br/>
**Default**: infinite
To limit how many pages will be crawled and audited:
```ruby
p instance.call( 'opts.link_count_limit=', 1 )
#=> 1
```
## #<a id="options_exclude_cookies" href="#options_exclude_cookies">Exclude cookies (opts.exclude_cookies=)</a>
**Expects**: Array of Strings <br/>
**Default**: []
To exclude cookies from the audit process:
```ruby
p instance.call( 'opts.exclude_cookies=', [ 'sessionid', 'some_auth_cookie' ] )
#=> ["sessionid", "some_auth_cookie"]
```
### [Exclude vectors (opts.exclude_vectors=)](#options_exclude_vectors)
**Expects**: Array of Strings <br/>
**Default**: []
To exclude input vectors from the audit process:
```ruby
p instance.call( 'opts.exclude_vectors=', [ 'password', 'csrf_token' ] )
#=> ["password", "csrf_token"]
```
### [User agent (opts.user_agent=)](#options_user_agent)
**Expects**: String <br/>
**Default**: Arachni/\<version\>
To set the user agent:
```ruby
p instance.call( 'opts.user_agent=', 'FooBar/0.1' )
#=> "FooBar/0.1"
```
### <a id="options_exclude" href="#options_exclude">Exclude patterns (opts.exclude=)</a>
**Expects**: Array of patterns (String or Regexp) <br/>
**Default**: []
To set an exclude rule:
```ruby
p instance.call( "opts.exclude=", ["do_not_follow"] )
#=> [/do_not_follow/]
```
URLs that match any of the patterns in the array will be ignored.
### <a id="options_include" href="#options_include">Include patterns (opts.include=)</a>
**Expects**: Array of patterns (String or Regexp) <br/>
**Default**: []
To set an include rule:
```ruby
p instance.call( "opts.include=", ["only_follow_me"] )
#=> [/only_follow_me/]
```
Only URLs that match any of the patterns in the array will be followed and audited.
### <a id="options_cookie_jar" href="#options_cookie_jar">Cookie jar (opts.cookie_jar=)</a>
**Expects**: Hash <br/>
**Default**: {}
To set the cookie jar:
```ruby
cookies = {
"userid" => 1,
"sessionid" => "fdfdfDDfsdfszdf"
}
p instance.call( "opts.cookie_jar=", cookies )
#=> {"userid"=>1, "sessionid"=>"fdfdfDDfsdfszdf"}
```
### <a id="options_http_req_limit" href="#options_http_req_limit">HTTP request limit (opts.http_req_limit=)</a>
**Expects**: Integer <br/>
**Default**: 20
To limit how many concurrent HTTP request are sent:
```ruby
p instance.call( "opts.http_req_limit=", 20 )
#=> 20
```
**Note**: If your scan seems unresponsive try lowering the limit. <br>
**Warning**: Given enough bandwidth and a high limit it could cause a DoS. Be
careful when setting this option too high, don"t kill your server.
### <a id="options_redundant" href="#options_redundant">Redundancy patterns (opts.redundant=)</a>
**Expects**: Array of Hashes <br/>
**Default**: []
```ruby
redundant = { "follow_me_3_times" => 3, /follow_me_5_times/ => 5 }
p instance.call( "opts.redundant=", redundant )
#=> {/follow_me_3_times/=>3, /follow_me_5_times/=>5}
```
<a id="plugins" href="#plugins">Load the plugins you want to run</a>
--------------------------------------------
**Expects**: Hash
**Default**: {}
```ruby
plugins = {
'proxy' => {}, # empty options
'autologin' => {
'url' => 'http://demo.testfire.net/bank/login.aspx',
'params' => 'uid=jsmith&passw=Demo1234',
'check' => 'MY ACCOUNT'
},
}
p instance.call( "plugins.load", plugins )
#=> ["proxy", "autologin"]
```
<a id="modules" href="#modules">Load the modules you want to run</a>
--------------------------------------------
**Expects**: Array of Strings <br/>
**Default**: []
```ruby
# to load all modules
# mods = [ '*' ]
mods = [ 'xss', 'path_traversal' ]
p instance.call( "modules.load", mods )
#=> ["xss", "path_traversal"]
```
<a id="run" href="#run">Run the framework</a>
-------------------------
To run the framework:
```ruby
instance.call( "framework.run" )
```
### [Get the progress while busy](#run_busy)
A handful of progress data can be requested by calling the
```framework.progress_data``` method.
This will include messages, statistics, discovered issues, status etc.<br/>
See:
[http://arachni.github.com/arachni/Arachni/RPC/Server/Framework.html#progress_data-instance_method](http://arachni.github.com/arachni/Arachni/RPC/Server/Framework.html#progress_data-instance_method)
```ruby
while instance.call( "framework.busy?" )
instance.call( "framework.progress_data" )['messages'].each do |out|
type, msg = *out.to_a.first
next if msg.empty?
puts "#{type}: #{msg}"
end
sleep 0.5
end
```
### <a id="run_pause_resume" href="#run_pause_resume">Pausing and Resuming</a>
```ruby
cnt = 0
while instance.call( "framework.busy?" )
instance.call( "service.output" ).each do |out|
type, msg = *out.to_a.first
next if msg.empty?
puts "#{type}: #{msg}"
end
if cnt % 2 == 0
puts 'Pausing...'
instance.call( "framework.pause!" )
else
puts 'Resuming...'
instance.call( "framework.resume!" )
end
sleep 0.5
cnt += 1
end
```
<a id="report" href="#report">Get the report</a>
-------------------------
To grab the results of the audit as a hash:
```ruby
instance.call( "framework.report" )
# to receive a YAML serialized hash -- may prevent type errors
#instance.call( "framework.serialized_report" )
```
To grab the results of the audit as an AuditStore object:
```ruby
instance.call( "framework.auditstore" )
# to receive a YAML serialized auditstore
#instance.call( "framework.serialized_auditstore" )
```
<a id="shutdown" href="#shutdown">Shutdown the server</a>
--------------------------------
To completely shutdown the server:
```ruby
instance.call( "service.shutdown" )
```
<a id="cancel" href="#cancel">Cancelling the scan</a>
------------------------------
In order to cancel a running scan it"s best to tell the Arachni instance
to clean up after itself before forcing a shutdown.<br/>
That way the framework and all running plugins will get a chance to
register their results.
```ruby
instance.call( "framework.clean_up" )
report = instance.call( "framework.report" )
# or
# report = instance.call( "framework.auditstore" )
instance.call( "service.shutdown" )
```
<a id="example" href="#example">A minimalistic example</a>
----------------------------------
```ruby
require 'arachni/rpc/pure'
require 'pp'
dispatcher = Arachni::RPC::Pure::Client.new(
host: 'localhost',
port: 7331
)
instance_info = dispatcher.call( "dispatcher.dispatch" )
host, port = instance_info['url'].split( ':' )
instance = Arachni::RPC::Pure::Client.new(
host: host,
port: port,
token: instance_info['token']
)
begin
opts = {
'url' => 'http://demo.testfire.net',
'audit_links' => true,
'audit_forms' => true,
'audit_cookies' => true,
# 'link_count_limit' => 1, # uncomment this line for a quick scan
}
# instance.call( 'modules.load', ['xss'] )
instance.call( 'modules.load_all' ) # comment this line and uncomment the above line for a quick scan
instance.call( 'opts.set', opts )
instance.call( 'framework.run' )
#
# wait until the framework is finished
#
# you can also request a report at any point during the scan to get results
# as they are logged but let's keep it simple for the example
#
print "Running."
while instance.call( 'framework.busy?' )
sleep 1
print '.'
end
puts '[Done]'
rescue
puts
puts 'Something bad happened.'
instance.call( "framework.clean_up" )
ensure
puts "Report:"
puts '--------------'
pp instance.call( 'framework.report' )
puts "Shutting down."
instance.call( 'service.shutdown' )
end
```