Thursday, 5 June 2008

Simple code for finding the number of whole days between two Java Calendars

So... here's a bit of code I've written that should calculate the number of whole days between tw0 Java calendars. Unit testing comes up perfectly, showing that it takes account of TimeZones and all that tom-foolery that I can think of with Java Calendar.

Of course, this wouldn't ever happen if you used Joda-Time, but not all of us are that lucky!

/**
* Returns the number of days between the start and end of the period. Partial days are discarded. Time zone information accounted for, so be careful
*
* @return The number of whole days between the two as a positive integer.
*/
public Integer differenceInDays() {
long start = period.getStart().getTimeInMillis();
long end = period.getEnd().getTimeInMillis();
long difference = Math.abs(end - start);
return new Integer((int) (difference / DateUtils.MILLIS_PER_DAY));
}
(DateUtils.MILLIS_PER_DAY is a constant, which is actually a long of size 86400000)

Friday, 4 April 2008

Java Time Zone 1.4 Reference

List of Java TimeZone IDs for Java 1.4.2.16

Printed by running TimeZone.getAvailableIDs()

Use at your own risk ;)

Etc/GMT+12
Etc/GMT+11
MIT
Pacific/Apia
Pacific/Midway
Pacific/Niue
Pacific/Pago_Pago
Pacific/Samoa
US/Samoa
America/Adak
America/Atka
Etc/GMT+10
HST
Pacific/Fakaofo
Pacific/Honolulu
Pacific/Johnston
Pacific/Rarotonga
Pacific/Tahiti
US/Aleutian
US/Hawaii
Pacific/Marquesas
AST
America/Anchorage
America/Juneau
America/Nome
America/Yakutat
Etc/GMT+9
Pacific/Gambier
US/Alaska
America/Dawson
America/Ensenada
America/Los_Angeles
America/Tijuana
America/Vancouver
America/Whitehorse
Canada/Pacific
Canada/Yukon
Etc/GMT+8
Mexico/BajaNorte
PST
PST8PDT
Pacific/Pitcairn
US/Pacific
US/Pacific-New
America/Boise
America/Cambridge_Bay
America/Chihuahua
America/Dawson_Creek
America/Denver
America/Edmonton
America/Hermosillo
America/Inuvik
America/Mazatlan
America/Phoenix
America/Shiprock
America/Yellowknife
Canada/Mountain
Etc/GMT+7
MST
MST7MDT
Mexico/BajaSur
Navajo
PNT
US/Arizona
US/Mountain
America/Belize
America/Cancun
America/Chicago
America/Costa_Rica
America/El_Salvador
America/Guatemala
America/Indiana/Knox
America/Indiana/Petersburg
America/Indiana/Vincennes
America/Knox_IN
America/Managua
America/Menominee
America/Merida
America/Mexico_City
America/Monterrey
America/North_Dakota/Center
America/North_Dakota/New_Salem
America/Rainy_River
America/Rankin_Inlet
America/Regina
America/Swift_Current
America/Tegucigalpa
America/Winnipeg
CST
CST6CDT
Canada/Central
Canada/East-Saskatchewan
Canada/Saskatchewan
Chile/EasterIsland
Etc/GMT+6
Mexico/General
Pacific/Easter
Pacific/Galapagos
US/Central
US/Indiana-Starke
America/Atikokan
America/Bogota
America/Cayman
America/Coral_Harbour
America/Detroit
America/Eirunepe
America/Fort_Wayne
America/Grand_Turk
America/Guayaquil
America/Havana
America/Indiana/Indianapolis
America/Indiana/Marengo
America/Indiana/Vevay
America/Indiana/Winamac
America/Indianapolis
America/Iqaluit
America/Jamaica
America/Kentucky/Louisville
America/Kentucky/Monticello
America/Lima
America/Louisville
America/Montreal
America/Nassau
America/New_York
America/Nipigon
America/Panama
America/Pangnirtung
America/Port-au-Prince
America/Porto_Acre
America/Resolute
America/Rio_Branco
America/Thunder_Bay
America/Toronto
Brazil/Acre
Canada/Eastern
Cuba
EST
EST5EDT
Etc/GMT+5
IET
Jamaica
US/East-Indiana
US/Eastern
US/Michigan
America/Anguilla
America/Antigua
America/Aruba
America/Asuncion
America/Barbados
America/Blanc-Sablon
America/Boa_Vista
America/Campo_Grande
America/Caracas
America/Cuiaba
America/Curacao
America/Dominica
America/Glace_Bay
America/Goose_Bay
America/Grenada
America/Guadeloupe
America/Guyana
America/Halifax
America/La_Paz
America/Manaus
America/Martinique
America/Moncton
America/Montserrat
America/Port_of_Spain
America/Porto_Velho
America/Puerto_Rico
America/Santiago
America/Santo_Domingo
America/St_Kitts
America/St_Lucia
America/St_Thomas
America/St_Vincent
America/Thule
America/Tortola
America/Virgin
Antarctica/Palmer
Atlantic/Bermuda
Atlantic/Stanley
Brazil/West
Canada/Atlantic
Chile/Continental
Etc/GMT+4
PRT
America/St_Johns
CNT
Canada/Newfoundland
AGT
America/Araguaina
America/Argentina/Buenos_Aires
America/Argentina/Catamarca
America/Argentina/ComodRivadavia
America/Argentina/Cordoba
America/Argentina/Jujuy
America/Argentina/La_Rioja
America/Argentina/Mendoza
America/Argentina/Rio_Gallegos
America/Argentina/San_Juan
America/Argentina/Tucuman
America/Argentina/Ushuaia
America/Bahia
America/Belem
America/Buenos_Aires
America/Catamarca
America/Cayenne
America/Cordoba
America/Fortaleza
America/Godthab
America/Jujuy
America/Maceio
America/Mendoza
America/Miquelon
America/Montevideo
America/Paramaribo
America/Recife
America/Rosario
America/Sao_Paulo
Antarctica/Rothera
BET
Brazil/East
Etc/GMT+3
America/Noronha
Atlantic/South_Georgia
Brazil/DeNoronha
Etc/GMT+2
America/Scoresbysund
Atlantic/Azores
Atlantic/Cape_Verde
Etc/GMT+1
Africa/Abidjan
Africa/Accra
Africa/Bamako
Africa/Banjul
Africa/Bissau
Africa/Casablanca
Africa/Conakry
Africa/Dakar
Africa/El_Aaiun
Africa/Freetown
Africa/Lome
Africa/Monrovia
Africa/Nouakchott
Africa/Ouagadougou
Africa/Sao_Tome
Africa/Timbuktu
America/Danmarkshavn
Atlantic/Canary
Atlantic/Faeroe
Atlantic/Faroe
Atlantic/Madeira
Atlantic/Reykjavik
Atlantic/St_Helena
Eire
Etc/GMT
Etc/GMT+0
Etc/GMT-0
Etc/GMT0
Etc/Greenwich
Etc/UCT
Etc/UTC
Etc/Universal
Etc/Zulu
Europe/Belfast
Europe/Dublin
Europe/Guernsey
Europe/Isle_of_Man
Europe/Jersey
Europe/Lisbon
Europe/London
GB
GB-Eire
GMT
GMT0
Greenwich
Iceland
Portugal
UCT
UTC
Universal
WET
Zulu
Africa/Algiers
Africa/Bangui
Africa/Brazzaville
Africa/Ceuta
Africa/Douala
Africa/Kinshasa
Africa/Lagos
Africa/Libreville
Africa/Luanda
Africa/Malabo
Africa/Ndjamena
Africa/Niamey
Africa/Porto-Novo
Africa/Tunis
Africa/Windhoek
Arctic/Longyearbyen
Atlantic/Jan_Mayen
CET
ECT
Etc/GMT-1
Europe/Amsterdam
Europe/Andorra
Europe/Belgrade
Europe/Berlin
Europe/Bratislava
Europe/Brussels
Europe/Budapest
Europe/Copenhagen
Europe/Gibraltar
Europe/Ljubljana
Europe/Luxembourg
Europe/Madrid
Europe/Malta
Europe/Monaco
Europe/Oslo
Europe/Paris
Europe/Podgorica
Europe/Prague
Europe/Rome
Europe/San_Marino
Europe/Sarajevo
Europe/Skopje
Europe/Stockholm
Europe/Tirane
Europe/Vaduz
Europe/Vatican
Europe/Vienna
Europe/Warsaw
Europe/Zagreb
Europe/Zurich
MET
Poland
ART
Africa/Blantyre
Africa/Bujumbura
Africa/Cairo
Africa/Gaborone
Africa/Harare
Africa/Johannesburg
Africa/Kigali
Africa/Lubumbashi
Africa/Lusaka
Africa/Maputo
Africa/Maseru
Africa/Mbabane
Africa/Tripoli
Asia/Amman
Asia/Beirut
Asia/Damascus
Asia/Gaza
Asia/Istanbul
Asia/Jerusalem
Asia/Nicosia
Asia/Tel_Aviv
CAT
EET
Egypt
Etc/GMT-2
Europe/Athens
Europe/Bucharest
Europe/Chisinau
Europe/Helsinki
Europe/Istanbul
Europe/Kaliningrad
Europe/Kiev
Europe/Mariehamn
Europe/Minsk
Europe/Nicosia
Europe/Riga
Europe/Simferopol
Europe/Sofia
Europe/Tallinn
Europe/Tiraspol
Europe/Uzhgorod
Europe/Vilnius
Europe/Zaporozhye
Israel
Libya
Turkey
Africa/Addis_Ababa
Africa/Asmara
Africa/Asmera
Africa/Dar_es_Salaam
Africa/Djibouti
Africa/Kampala
Africa/Khartoum
Africa/Mogadishu
Africa/Nairobi
Antarctica/Syowa
Asia/Aden
Asia/Baghdad
Asia/Bahrain
Asia/Kuwait
Asia/Qatar
Asia/Riyadh
EAT
Etc/GMT-3
Europe/Moscow
Europe/Volgograd
Indian/Antananarivo
Indian/Comoro
Indian/Mayotte
W-SU
Asia/Riyadh87
Asia/Riyadh88
Asia/Riyadh89
Mideast/Riyadh87
Mideast/Riyadh88
Mideast/Riyadh89
Asia/Tehran
Iran
Asia/Baku
Asia/Dubai
Asia/Muscat
Asia/Tbilisi
Asia/Yerevan
Etc/GMT-4
Europe/Samara
Indian/Mahe
Indian/Mauritius
Indian/Reunion
NET
Asia/Kabul
Asia/Aqtau
Asia/Aqtobe
Asia/Ashgabat
Asia/Ashkhabad
Asia/Dushanbe
Asia/Karachi
Asia/Oral
Asia/Samarkand
Asia/Tashkent
Asia/Yekaterinburg
Etc/GMT-5
Indian/Kerguelen
Indian/Maldives
PLT
Asia/Calcutta
Asia/Colombo
IST
Asia/Katmandu
Antarctica/Mawson
Antarctica/Vostok
Asia/Almaty
Asia/Bishkek
Asia/Dacca
Asia/Dhaka
Asia/Novosibirsk
Asia/Omsk
Asia/Qyzylorda
Asia/Thimbu
Asia/Thimphu
BST
Etc/GMT-6
Indian/Chagos
Asia/Rangoon
Indian/Cocos
Antarctica/Davis
Asia/Bangkok
Asia/Hovd
Asia/Jakarta
Asia/Krasnoyarsk
Asia/Phnom_Penh
Asia/Pontianak
Asia/Saigon
Asia/Vientiane
Etc/GMT-7
Indian/Christmas
VST
Antarctica/Casey
Asia/Brunei
Asia/Chongqing
Asia/Chungking
Asia/Harbin
Asia/Hong_Kong
Asia/Irkutsk
Asia/Kashgar
Asia/Kuala_Lumpur
Asia/Kuching
Asia/Macao
Asia/Macau
Asia/Makassar
Asia/Manila
Asia/Shanghai
Asia/Singapore
Asia/Taipei
Asia/Ujung_Pandang
Asia/Ulaanbaatar
Asia/Ulan_Bator
Asia/Urumqi
Australia/Perth
Australia/West
CTT
Etc/GMT-8
Hongkong
PRC
Singapore
Australia/Eucla
Asia/Choibalsan
Asia/Dili
Asia/Jayapura
Asia/Pyongyang
Asia/Seoul
Asia/Tokyo
Asia/Yakutsk
Etc/GMT-9
JST
Japan
Pacific/Palau
ROK
ACT
Australia/Adelaide
Australia/Broken_Hill
Australia/Darwin
Australia/North
Australia/South
Australia/Yancowinna
AET
Antarctica/DumontDUrville
Asia/Sakhalin
Asia/Vladivostok
Australia/ACT
Australia/Brisbane
Australia/Canberra
Australia/Currie
Australia/Hobart
Australia/Lindeman
Australia/Melbourne
Australia/NSW
Australia/Queensland
Australia/Sydney
Australia/Tasmania
Australia/Victoria
Etc/GMT-10
Pacific/Guam
Pacific/Port_Moresby
Pacific/Saipan
Pacific/Truk
Pacific/Yap
Australia/LHI
Australia/Lord_Howe
Asia/Magadan
Etc/GMT-11
Pacific/Efate
Pacific/Guadalcanal
Pacific/Kosrae
Pacific/Noumea
Pacific/Ponape
SST
Pacific/Norfolk
Antarctica/McMurdo
Antarctica/South_Pole
Asia/Anadyr
Asia/Kamchatka
Etc/GMT-12
Kwajalein
NST
NZ
Pacific/Auckland
Pacific/Fiji
Pacific/Funafuti
Pacific/Kwajalein
Pacific/Majuro
Pacific/Nauru
Pacific/Tarawa
Pacific/Wake
Pacific/Wallis
NZ-CHAT
Pacific/Chatham
Etc/GMT-13
Pacific/Enderbury
Pacific/Tongatapu
Etc/GMT-14
Pacific/Kiritimati

Wednesday, 1 August 2007

Restricting the search scope to improve performance

One major issue with DeepWeaver is that any call that results in the body predicate executing results in substantial slowdown, particularly upon a large code base. Predicates that result in body being called include call(x,y,z) and assign(x,y,z) when x is a free variable.

In both cases here, x represents a statement, and without that statement being bound, the body predicate must be called to traverse all Units in the codebase to check whether it matches your criteria. This can be particularly slow if the matching criteria itself is expensive.

Remember DeepWeaver doesn't just analyse the class or package you specified, if you call an unbound assign or call predicate it will dive into the libraries and look for results. This includes looking through classes such as Object and Throwable from which common Java features are derived. Needless to say, it's going to slow you down.

At time of writing there's no artificial predicate to limit the scope based on what is specified in your command line, but there's an easy way to do this manually.
If you want to restrict yourself to a certain class or set of classes, simply specify:
scope(u)=class(c,"org.owasp.webgoat.session.User*"), units(body(sootmethod(c,<-),<-),u);

and then
scope(u), assign(u,y,z), ...

and now the first predicate in assign is bound to be a unit from some class beginning with User in the specified package. This dramatically restricts the scope of possible values for u and thus greatly improves the resulting performance.

Notice the use of wildcards here, you may use wildcards in both the class and package name but you must attempt to specify the class name to some degree. So you could say org.owasp.*.User* to find your classes, or ever org.*.* but that dot is essential to include, or it will assume your package to be a class name and fail to find the result you're looking for.

Tuesday, 31 July 2007

Datatypes in DeepWeaver

DeepWeaver introduces limited type safety, as previously described.
To use types, you can use either built in types, or any other Java object by specifying it's full long-form name (eg. java.awt.AbstractListener)
If you're feeling really brave you can add your own datatype (although I've never seen a need to do so), just add it to the list in dw.type.Types and then add your own corresponding class file.

Here's a brief run-down of the built-in types:
  • String: Letters in double-quotes. Usually used for matching, printing or restriction
  • List: A Prolog style list, but shorthand list notation is not yet available
  • Stmt: Corresponds with the soot Unit type. A line or unit of code, which also retains information about the methods and values associated with it
  • Value: Corresponds with the soot Value type. A code value that also stores information about its declaring unit (but may be linked to by more than one unit!)
  • Block: A CodeBlock, or series of units, usually in some order that relates to their exection. In fact, it is a implemented as a chain of units, and the unit(cb,u) predicate can be used to split it into these component units. Also stores information about its parent method, and synchronization
  • Method: An analysis of an entire Method, including information of body, declaring class and synchronization.
  • Class: An analysis of an entire class, corresponding with soot.SootClass.

Tuesday, 24 July 2007

Running DeepWeaver from the command line

Calling DeepWeaver from the command line is pretty similar to any other Java program, ie. not very pretty! The basic format is:
java -cp CLASSPATH dw.DW script.dw org.mytarget.Target

where
  • CLASSPATH contains all the JARs required by DeepWeaver (these may be contained in the $CLASSPATH environment variable if you prefer) - should be separated by a colon. For more info, run java -help
  • script.dw is the name of the predicate script you want to run
  • org.mytarget.Target is the name of the class file on which you want to run it, and you can specify as many targets as you like
Commonly used paramaters are
  • -javac "javac -cp ~/workspace/dw/build/classes"
    • Instructs the use custom java compiler parameters, which is nearly always necessary. In this case the java compiler will add my classes directory to its classpath.
  • -addcp /home/user/lib
    • Adds a custom class path for inclusion in Deepweaver execution. All the libraries you need, if they're not directly locatable from your current execution directory need to be added. This should point to either a root folder (eg. looking for org.apache.catalina.Role then it must be in /home/user/lib/org/apache/catalina/Role.class) or a JAR file. The easy way around this, if you have lots of JAR files is to create a library folder and just inflate all your JAR files there, that way you only ever have to specify one -addcp parameter
  • -time
    • displays timing information
  • -v
    • verbose output
You can also specify targets for execution en masse by using the + symbol on pacakages
+org.apache.catalina

Anything linked to code that is analysed will be loaded dynamically if it can be found.

Confused? Here's a sample command for Linux
java -cp jars/antlr-2.7.5.jar:jars/sootclasses-2534-1.jar:jars/jasminclasses-2327.jar:jars/polyglotclasses-1.3.2.jar:build/classes dw.DW -javac "javac -cp ${workspace_loc}/dw/build/classes" -addcp ~/workspace/webgoat_src/build/WEB-INF/classes/ -addcp ~/jars transact.dw +org.owasp.webgoat

This isn't as bad as it may seem!
  • Our classpath requires 4 DW jar files and the build/classes folder, which we've specified using -cp (Eclipse adds this automatically if you run using it).
  • -javac tells the java compiler to look inside /dw/build/classes for compile info
  • The -addcp parameters add the webgoat compiled classes and library of extracted JAR files to the path
  • transact.dw is the script to be called on the source code
  • We want to analyse all code within org.owasp.webgoat

Monday, 23 July 2007

How to use the between predicate

The between predicate is an invaluable tool in your troubleshooting arsenal but it can also be quite tricky to use. The format of the predicate is:
between(a,z,x,b)

a and z should be code units that represent the start and end of the between block.
x is the code between a and z. If x is output, it will come in the form of one line of soot analysis at at time, in the form of units. If x is input, it should be a unit (or unit box) of one or more lines of code.
b is a boolean choice that selects between must (true) and may (false) analysis, eg. code in an if block between a and z may not be called between a and z and thus will be included in a may, but not a must analysis.

To use between to find the x variable, the most important thing is to ensure that you specify a and z as two non-equal code units. However, units are not necessarily the most instinctive way to specify a and z. For example, you may want to specify a or z in terms of a call. The wrong way to do this is:
between(call(<-,*,*), z, X, false)

This has a number of faults including:
  • There's nothing to stop the result of your call being the same as z (an arbitrary unit) which could give a null exception
  • Your input is likely to give multiple locations, which will give multiple results from between that may not be easily distinguishable from each other
Between is very powerful but you need to be firm about your input to it. Here is an example of how to use between to find all the code that may be between two method calls, one called begin, the other called commit.
getUnit(call(<-,p,*), y), getUnit(call(<-,q,*), z), methodMatches(p, "* begin(..)), methodMatches(q, "* commit(..)"), between(y, z, X, false )

This works because call's middle parameter returns a method. This method name can then be checked with methodMatches to ensure that it has an expected name. Since the two match parameters are different, this also ensures the two results won't be equal. Then we get the unit from the result so that we are sure between will be receiving a unit as both parameters. X is the result which may be multiple lines of code we can then test.

Thursday, 19 July 2007

Predicate Specification

A couple of notes on how to specify your predicates in DeepWeaver, that are worth mentioning because they're different from how they're specified in Prolog.

When you're specifiying a predicate there are a number of reserved keywords, and an few restrictions.
name(fn,sn)=getName(fn,sn);
This is the basic standard specification, fn and sn are bound or unbound variables of any type.
name(in fn, out sn)=getName(fn,sn);
This restricts the standard specification, because fn must be bound and sn must be unbound when this predicate is called, otherwise it will be ignored. You could overload is by following it with the first example above.
name(String fn, out sn)=getName(fn,sn);
Now fn must be a String object, but may be bound or unbound, sn must still be unbound.

Points of note:
  • Unless your variable name is one char long, it should start with a small letter or it will be mistaken for a type binding
  • in and out should not be used as variable names
  • Predicates can be overloaded, but they are overloaded in order (same as Prolog), meaning that if you have a predicate which has no binding or type restrictions, it should be the final specified predicate
  • No error will be thrown if your bindings cause a predicate call that you did not expect