Important!

This is a COPY of Massimo Pierro's web2py book, and it is only intended for my personal use.

The original book can and should be read here: https://web2py.com/books/default/chapter/29

This site has only one purpose: To keep the whole book on one big easy searchable html page.

What follows is NOT written by my - but by Massimo Pierro

web2py was launched in 2007 and now, after four years of continuous development, we have reached a very much needed fourth edition of this book. During this time, web2py has managed to win the affection of thousands of knowledgeable users and more than one hundred developers. Our collective effort has created one of the most full-featured Open Source Web Frameworks in existence.

I originally started web2py as a teaching tool because, I believe, the ability to build high quality web applications is of critical importance for the growth of a free and open society. It prevents the biggest players from monopolizing the flow of information. This motivation continues to be valid and it is even more important today.

In general, the purpose of any web framework is to make web development easier, faster and prevent developers from making mistakes, especially in matters related to security. In web2py we address these issues with our three main goals:

Ease of use is the primary goal for web2py. For us, this means reducing the learning and deployment time. This is why web2py is a full-stack framework without dependencies. It requires no installation and has no configuration files. Everything works out of the box, including a web server, database and a web-based IDE that gives access to all the main features. The API includes just 12 core objects, which are easy to work with and memorize. It can interoperate with most web servers, database and all Python libraries.

Faster development is the secondary goal. Every function of web2py has a default behavior (which can be overridden). For example, as soon as you have specified your data models, you will have access to a web-based database administration panel. web2py also generates automatically forms for your data and it allows you to easily expose the data in HTML, XML, JSON, RSS, etc.

Security is at the heart of web2py, and our goal here is to lock everything down to keep your systems and data safe. Therefore, our database layer eliminates SQL Injections. The template language prevents Cross Site Scripting vulnerabilities. The forms generated by web2py provide field validation and block Cross Site Request Forgeries. Passwords are always stored hashed. Sessions are stored server-side by default to prevent Cookie Tampering and session cookies are uuid to prevent Cookie Stealing.

web2py has always been built from the user perspective and is constantly optimized internally to become faster and leaner, whilst always maintaining backward compatibility.

web2py is free for you to use. If you benefit from it, we hope you will feel a little more like contributing back to society in whatever form you choose.

In 2011 InfoWorld magazine reviewed six of the most popular full-stack Python based web frameworks and ranked web2py highest. Also in 2011, web2py won the Bossie Award for best Open Source Development Software.

Introduction

web2py[web2py] is a free, open-source web framework for agile development of secure database-driven web applications; it is written in Python[python] and programmable in Python. web2py is a full-stack framework, meaning that it contains all the components you need to build fully functional web applications.

web2py is designed to guide a web developer to follow good software engineering practices, such as using the Model View Controller (MVC) pattern. web2py separates the data representation (the model) from the data presentation (the view) and also from the application logic and workflow (the controller). web2py provides libraries to help the developer design, implement, and test each of these three parts separately, and makes them work together.

web2py is built for security. This means that it automatically addresses many of the issues that can lead to security vulnerabilities, by following well established practices. For example, it validates all input (to prevent injections), escapes all output (to prevent cross-site scripting), renames uploaded files (to prevent directory traversal attacks). web2py leaves little choice to application developers in matters related to security.

web2py includes a Database Abstraction Layer (DAL) that writes SQL[sql:w] dynamically so that you, the developer, don't have to. The DAL knows how to generate SQL transparently for SQLite[sqlite], MySQL[mysql], PostgreSQL[postgres], MSSQL[mssql], FireBird[firebird], Oracle[oracle], IBM DB2[db2], Informix[informix], and Ingres[ingresdb]. The DAL can also generate function calls for the Google Datastore when running on the Google App Engine (GAE)[gae]. Experimentally we support more databases. Please check on the web2py web site and mailing list for more recent adapters. Once one or more database tables are defined, web2py also generates a fully functional web-based database administration interface to access the database and the tables.

web2py differs from other web frameworks in that it is the only framework to fully embrace the Web 2.0 paradigm, where the web is the computer. In fact, web2py does not require installation or configuration; it runs on any architecture that can run Python (Windows, Windows CE, Mac OS X, iOS, and Unix/Linux), and the development, deployment, and maintenance phases for the applications can be done via a local or remote web interface. web2py runs with CPython (the C implementation) and Jython (the Java implementation), on Python versions 2.4, 2.5, 2.6, and 2.7, although "officially" it only supports 2.5 so that we can guarantee backward compatibility for applications.

web2py provides a ticketing system. If an error occurs, a ticket is issued to the user, and the error is logged for the administrator.

web2py is open source and released under the LGPL version 3 license.

Another feature of web2py is that we, its developers, commit to maintain backward compatibility in future versions. We have done so since the first release of web2py in October, 2007. New features have been added and bugs have been fixed, but if a program worked with web2py 1.0, that program will still work today.

Here are some examples of web2py statements that illustrate its power and simplicity. The following code:

1.
db.define_table('person', Field('name'), Field('image', 'upload'))

creates a database table called "person" with two fields: "name", a string; and "image", something that needs to be uploaded (the actual image). If the table already exists but does not match this definition, it is altered appropriately.

Given the table defined above, the following code:

1.
form = crud.create(db.person)

creates an insert form for this table that allows users to upload images. It also validates a submitted form, renames the uploaded image in a secure way, stores the image in a file, inserts the corresponding record in the database, prevents double submission, and eventually modifies the form itself by adding error messages if the data submitted by the user does not pass validation.

The following code:

1.
2.
@auth.requires_permission('read','person')
def f(): ....

prevents visitors from accessing the function f unless the visitor is a member of a group whose members have permissions to "read" records of table "person". If the visitor is not logged in, he gets directed to a login page (provided by default by web2py).

The following code embeds a page component.

1.
{{=LOAD('other_controller','function.load',ajax=True, ajax_trap=True)}}

This instructs web2py to load in a view the content generated by the other controller function (this works with any function). It loads the content via Ajax, embeds it into the current page (using the current layout, not the layout of the other_controller function), and traps all forms contained in the loaded content so that they are also submitted via Ajax without reloading the page. It can also LOAD content from non-web2py applications.

The LOAD helper allows very modular design of applications; it is discussed in some detail in the last chapter of this book.

Principles

Python programming typically follows these basic principles:

web2py fully embraces the first two principles by forcing the developer to use sound software engineering practices that discourage repetition of code. web2py guides the developer through almost all the tasks common in web application development (creating and processing forms, managing sessions, cookies, errors, etc.).

request
web2py differs from other frameworks with regard to the third principle, which sometimes conflicts with the other two. In particular, web2py does not import user applications, but executes them in a predefined context. This context exposes the Python keywords, as well as the web2py keywords.

To some this may appear as magic, but it should not. Simply, in practice, some modules are already imported without you doing so. web2py is trying to avoid the annoying characteristic of other frameworks that force the developer to import the same modules at the top of every model and controller.

web2py, by importing its own modules, saves time and prevents mistakes, thus following the spirit of "don't repeat yourself" and "there should be only one way of doing things".

If the developer wishes to use other Python modules or third-party modules, those modules must be imported explicitly, as in any other Python program.

Web frameworks

PHP
ASP
JSP

At its most fundamental level, a web application consists of a set of programs (or functions) that are executed when the corresponding URL is visited. The output of the program is returned to the visitor and rendered by the browser.

The purpose of web frameworks is to allow developers to build new apps quickly, easily and without mistakes. This is done by providing APIs and tools that reduce and simplify the amount of coding that is required.

The two classic approaches for developing web applications are:

The first model is the one that was followed, for example, by early CGI scripts. The second model is followed, for example, by PHP[php] (where the code is in PHP, a C-like language), ASP (where the code is in Visual Basic), and JSP (where the code is in Java).

Here is an example of a PHP program that, when executed, retrieves data from a database and returns an HTML page showing the selected records:

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
<html><body><h1>Records</h1><?
mysql_connect(localhost,username,password);
@mysql_select_db(database) or die( "Unable to select database");
$query="SELECT * FROM contacts";
$result=mysql_query($query);
mysql_close();
$i=0;
while ($i < mysql_numrows($result)) {
$name=mysql_result($result,$i,"name");
$phone=mysql_result($result,$i,"phone");
echo "<b>$name</b><br>Phone:$phone<br /><br /><hr /><br />";
$i++;
}
?></body></html>

The problem with this approach is that code is embedded into HTML, but the very same code also needs to generate additional HTML and to generate SQL statements to query the database, entangling multiple layers of the application and making it difficult to read and maintain. The situation is even worse for Ajax applications, and the complexity grows with the number of pages (files) that make up the application.

The functionality of the above example can be expressed in web2py with two lines of Python code:

1.
2.
def index():
return HTML(BODY(H1('Records'), db().select(db.contacts.ALL)))

In this simple example, the HTML page structure is represented programmatically by the HTML, BODY, and H1 objects; the database db is queried by the select command; finally, everything is serialized into HTML. Notice that db is not a keyword but a user defined variable. We will use this name consistently to refer to a database connection to avoid confusion.

Web frameworks are typically categorized as one of two types: A "glued" framework is built by assembling (gluing together) several third-party components. A "full-stack" framework is built by creating components designed specifically to be tightly integrated and work together.

web2py is a full-stack framework. Almost all of its components are built from scratch and are designed to work together, but they function just as well outside of the complete web2py framework. For example, the Database Abstraction Layer (DAL) or the template language can be used independently of the web2py framework by importing gluon.dal or gluon.template into your own Python applications. gluon is the name of the web2py module that contains system libraries. Some web2py libraries, such as building and processing forms from database tables, have dependencies on other portions of web2py. web2py can also work with third-party Python libraries, including other template languages and DALs, but they will not be as tightly integrated as the original components.

Model-View-Controller

Model-View-Controller
web2py encourages the developer to separate data representation (the model), data presentation (the view) and the application workflow (the controller). Let's consider again the previous example and see how to build a web2py application around it. Here is an example of the web2py MVC edit interface:

image

The typical workflow of a request in web2py is described in the following diagram:

image

In the diagram:

It can serve and stream static files if the web server is not doing it already.

Here is a minimal and complete MVC application, consisting of three files:

"db.py" is the model:
1.
2.
3.
4.
db = DAL('sqlite://storage.sqlite')
db.define_table('contact',
Field('name'),
Field('phone'))

It connects to the database (in this example a SQLite database stored in the storage.sqlite file) and defines a table called contact. If the table does not exist, web2py creates it and, transparently and in the background, generates SQL code in the appropriate SQL dialect for the specific database engine used. The developer can see the generated SQL but does not need to change the code if the database back-end, which defaults to SQLite, is replaced with MySQL, PostgreSQL, MSSQL, FireBird, Oracle, DB2, Informix, Interbase, Ingres, and the Google App Engine (both SQL and NoSQL).

Once a table is defined and created, web2py also generates a fully functional web-based database administration interface, called appadmin, to access the database and the tables.

"default.py" is the controller:
1.
2.
3.
def contacts():
grid=SQLFORM.grid(db.contact, user_signature=False)
return locals()

In web2py, URLs are mapped to Python modules and function calls. In this case, the controller contains a single function (or "action") called contacts. An action may return a string (the returned web page) or a Python dictionary (a set of key:value pairs) or the set of local variables (as in this example). If the function returns a dictionary, it is passed to a view with the same name as the controller/function, which in turn renders the page. In this example, the function contacts generates a select/search/create/update/delete grid for table db.contact and returns the grid to the view.

"default/contacts.html" is the view:
1.
2.
3.
{{extend 'layout.html'}}
<h1>Manage My Contacts</h1>
{{=grid}}

This view is called automatically by web2py after the associated controller function (action) is executed. The purpose of this view is to render the variables in the returned dictionary (in our case grid) into HTML. The view file is written in HTML, but it embeds Python code delimited by the special {{ and }} delimiters. This is quite different from the PHP code example, because the only code embedded into the HTML is "presentation layer" code. The "layout.html" file referenced at the top of the view is provided by web2py and constitutes the basic layout for all web2py applications. The layout file can easily be modified or replaced.

Why web2py

web2py is one of many web application frameworks, but it has compelling and unique features. web2py was originally developed as a teaching tool, with the following primary motivations:

WSGI [wsgi:w,wsgi:o] (Web Server Gateway Interface) is an emerging Python standard for communication between a web server and Python applications).

Here is a screenshot of the main web2py admin interface:

image

Security

security
The Open Web Application Security Project[owasp] (OWASP) is a free and open worldwide community focused on improving the security of application software.

OWASP has listed the top ten security issues that put web applications at risk. That list is reproduced here, along with a description of how each issue is addressed by web2py:

web2py was reviewed for security and you can find the result of the review in ref.[pythonsecurity].

In the box

You can download web2py from the official web site:

1.
http://www.web2py.com

web2py is composed of the following components:

web2py is distributed in source code, and in binary form for Microsoft Windows and for Mac OS X.

The source code distribution can be used in any platform where Python runs and includes the above-mentioned components. To run the source code, you need Python 2.5 pre-installed on the system. You also need one of the supported database engines installed. For testing and light-demand applications, you can use the SQLite database, included with Python 2.5.

The binary versions of web2py (for Windows and Mac OS X) include a Python 2.5 interpreter and the SQLite database. Technically, these two are not components of web2py. Including them in the binary distributions enables you to run web2py out of the box.

The following image depicts the overall web2py structure:

image

License

license

web2py is licensed under the LGPL version 3 License. The full text of the license if available in ref.[lgpl3].

In accordance with LGPL you may:

Yet you must:

The license includes the usual disclaimer:

THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM “AS IS” WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.

IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Earlier versions

Earlier versions of web2py, 1.0.*-1.90.*, were released under the GPL2 license plus a commercial exception which, for practical purposes, was very similar to the current LPGLv3.

Third party software distributed with web2py

web2py contains third party software under the gluon/contrib/ folder and various JavaScript and CSS files. These files are distributed with web2py under their original licenses, as stated in the files.

Acknowledgments

web2py was originally developed by and copyrighted by Massimo Di Pierro. The first version (1.0) was released in October, 2007. Since then it has been adopted by many users, some of whom have also contributed bug reports, testing, debugging, patches, and proofreading of this book.

Some of the major contributors are, in alphabetical order by first name:

Alexey Nezhdanov, Alvaro Justen, Andrew Willimott, Angelo Compagnucci, Anthony Bastardi, Antonio Ramos, Arun K. Rajeevan, Attila Csipa, Bill Ferret, Boris Manojlovic, Branko Vukelic, Brian Meredyk, Bruno Rocha, Carlos Galindo, Carsten Haese, Chris Clark, Chris Steel, Christian Foster Howes, Christopher Smiga, CJ Lazell, Cliff Kachinske, Craig Younkins, Daniel Lin, David Harrison, David Wagner, Denes Lengyel, Douglas Soares de Andrade, Eric Vicenti, Falko Krause, Farsheed Ashouri, Fran Boon, Francisco Gama, Fred Yanowski, Gilson Filho, Graham Dumpleton, Gyuris Szabolcs, Hamdy Abdel-Badeea, Hans Donner, Hans Murx, Hans C. v. Stockhausen, Ian Reinhart Geiser, Ismael Serratos, Jan Beilicke, Jonathan Benn, Jonathan Lundell, Josh Goldfoot, Jose Jachuf, Josh Jaques, José Vicente de Sousa, Keith Yang, Kenji Hosoda, Kyle Smith, Limodou, Lucas D'Ávila, Marcel Leuthi, Marcel Hellkamp, Marcello Della Longa, Mariano Reingart, Mark Larsen, Mark Moore, Markus Gritsch, Martin Hufsky, Martin Mulone, Mateusz Banach, Miguel Lopez, Michael Willis, Michele Comitini, Nathan Freeze, Niall Sweeny, Niccolo Polo, Nicolas Bruxer, Olaf Ferger, Omi Chiba, Ondrej Such, Ovidio Marinho Falcao Neto, Pai, Paolo Caruccio, Patrick Breitenbach, Phyo Arkar Lwin, Pierre Thibault, Ramjee Ganti, Robin Bhattacharyya, Ross Peoples, Ruijun Luo, Ryan Seto, Scott Roberts, Sergey Podlesnyi, Sharriff Aina, Simone Bizzotto, Sriram Durbha, Sterling Hankins, Stuart Rackham, Telman Yusupov, Thadeus Burgess, Tim Michelsen, Timothy Farrell, Yair Eshel, Yarko Tymciurak, Younghyun Jo, Vidul Nikolaev Petrov, Vinicius Assef, Zahariash.

I am sure I forgot somebody, so I apologize.

I particularly thank Jonathan, Mariano, Bruno, Martin, Nathan, Simone, Thadeus, Tim, Iceberg, Denes, Hans, Christian, Fran and Patrick for their major contributions to web2py and Anthony, Alvaro, Bruno, Denes, Felipe, Graham, Jonathan, Hans, Kyle, Mark, Michele, Richard, Robin, Roman, Scott, Shane, Sharriff, Sriram, Sterling, Stuart, Thadeus (and others) for proofreading various versions of this book. Their contribution was invaluable. If you find any errors in this book, they are exclusively my fault, probably introduced by a last-minute edit. I also thank Ryan Steffen of Wiley Custom Learning Solutions for help with publishing the first edition of this book.

web2py contains code from the following authors, whom I would like to thank:

Guido van Rossum for Python[python], Peter Hunt, Richard Gordon, Timothy Farrell for the Rocket[rocket] web server, Christopher Dolivet for EditArea[editarea], Bob Ippolito for simplejson[simplejson], Simon Cusack and Grant Edwards for pyRTF[pyrtf], Dalke Scientific Software for pyRSS2Gen[pyrss2gen], Mark Pilgrim for feedparser[feedparser], Trent Mick for markdown2[markdown2], Allan Saddi for fcgi.py, Evan Martin for the Python memcache module[memcache], John Resig for jQuery[jquery].

The cover of this book was designed by Peter Kirchner at Young Designers.

I thank Helmut Epp (provost of DePaul University), David Miller (Dean of the College of Computing and Digital Media of DePaul University), and Estia Eichten (Member of MetaCryption LLC), for their continuous trust and support.

Finally, I wish to thank my wife, Claudia, and my son, Marco, for putting up with me during the many hours I have spent developing web2py, exchanging emails with users and collaborators, and writing this book. This book is dedicated to them.

About this book

This book includes the following chapters, besides this introduction:

This book only covers basic web2py functionalities and the API that ships with web2py. This book does not cover web2py appliances (i.e. ready made applications).

You can download web2py appliances from the corresponding web site [appliances].

You can find additional topics discussed on AlterEgo[alterego], the interactive web2py FAQ.

This book has been written using the markmin syntax

MARKMIN
and automatically converted to HTML, LaTeX and PDF.

Elements of style

PEP8 [style] contains good style practices when programming with Python. You will find that web2py does not always follow these rules. This is not because of omissions or negligence; it is our belief that the users of web2py should follow these rules and we encourage it. We chose not to follow some of those rules when defining web2py helper objects in order to minimize the probability of name conflict with objects defined by the user.

For example, the class that represents a <div> is called DIV, while according to the Python style reference it should have been called Div. We believe that, for this specific example that using an all-upper-case "DIV" is a more natural choice. Moreover, this approach leaves programmers free to create a class called "Div" if they choose to do so. Our syntax also maps naturally into the DOM notation of most browsers (including, for example, Firefox).

According to the Python style guide, all-upper-case strings should be used for constants and not variables. Continuing with our example, even considering that DIV is a class, it is a special class that should never be modified by the user because doing so would break other web2py applications. Hence, we believe this qualifies the DIV class as something that should be treated as a constant, further justifying our choice of notation.

In summary, the following conventions are followed:

In all other cases we believe we have followed, as much as possible, the Python Style Guide (PEP8). For example all instance objects are lower-case (request, response, session, cache), and all internal classes are capitalized.

In all the examples of this book, web2py keywords are shown in bold, while strings and comments are shown in italic.

The Python language

Python

About Python

Python is a general-purpose high-level programming language. Its design philosophy emphasizes programmer productivity and code readability. It has a minimalist core syntax with very few basic commands and simple semantics, but it also has a large and comprehensive standard library, including an Application Programming Interface (API)

API
to many of the underlying operating system (OS) functions. Python code, while minimalist, defines built-in objects such as linked lists (list), tuples (tuple), hash tables (dict), and arbitrarily long integers (long).

Python supports multiple programming paradigms, including object-oriented (class), imperative (def), and functional (lambda) programming. Python has a dynamic type system and automatic memory management using reference counting (similar to Perl, Ruby, and Scheme).

Python was first released by Guido van Rossum in 1991. The language has an open, community-based development model managed by the non-profit Python Software Foundation. There are many interpreters and compilers that implement the Python language, including one in Java (Jython) but, in this brief review, we refer to the reference C implementation created by Guido.

You can find many tutorials, the official documentation and library references of the language on the official Python website.[python]

For additional Python references, we can recommend the books in ref.[guido] and ref.[lutz] .

You may skip this chapter if you are already familiar with the Python language.

Starting up

shell
The binary distributions of web2py for Microsoft Windows or Apple OS X come packaged with the Python interpreter built into the distribution file itself.

You can start it on Windows with the following command (type at the DOS prompt):

1.
web2py.exe -S welcome

On Apple OS X, enter the following command type in a Terminal window (assuming you're in the same folder as web2py.app):

1.
./web2py.app/Contents/MacOS/web2py -S welcome

On a Linux or other Unix box, chances are that you have Python already installed. If so, at a shell prompt type:

1.
python web2py.py -S welcome

If you do not have Python 2.5 (or later 2.x) already installed, you will have to download and install it before running web2py.

The -S welcome command line option instructs web2py to run the interactive shell as if the commands were executed in a controller for the welcome application, the web2py scaffolding application. This exposes almost all web2py classes, objects and functions to you. This is the only difference between the web2py interactive command line and the normal Python command line.

The admin interface also provides a web-based shell for each application. You can access the one for the "welcome" application at.

1.
http://127.0.0.1:8000/admin/shell/index/welcome

You can try all the examples in this chapter using the normal shell or the web-based shell.

help, dir

help
dir

The Python language provides two commands to obtain documentation about objects defined in the current scope, both built-in and user-defined.

We can ask for help about an object, for example "1":

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
>>> help(1)
Help on int object:

class int(object)
|
int(x[, base]) -> integer
|
|
Convert a string or number to an integer, if possible. A floating point
| argument will be truncated towards zero (this does not include a string
| representation of a floating point number!) When converting a string, use
| the optional base. It is an error to supply a base when converting a
| non-string. If the argument is outside the integer range a long object
| will be returned instead.
|
|
Methods defined here:
|
|
__abs__(...)
|
x.__abs__() <==> abs(x)
...

and, since "1" is an integer, we get a description about the int class and all its methods. Here the output has been truncated because it is very long and detailed.

Similarly, we can obtain a list of methods of the object "1" with the command dir:

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
>>> dir(1)
[
'__abs__', '__add__', '__and__', '__class__', '__cmp__', '__coerce__',
'__delattr__', '__div__', '__divmod__', '__doc__', '__float__',
'__floordiv__', '__getattribute__', '__getnewargs__', '__hash__', '__hex__',
'__index__', '__init__', '__int__', '__invert__', '__long__', '__lshift__',
'__mod__', '__mul__', '__neg__', '__new__', '__nonzero__', '__oct__',
'__or__', '__pos__', '__pow__', '__radd__', '__rand__', '__rdiv__',
'__rdivmod__', '__reduce__', '__reduce_ex__', '__repr__', '__rfloordiv__',
'__rlshift__', '__rmod__', '__rmul__', '__ror__', '__rpow__', '__rrshift__',
'__rshift__', '__rsub__', '__rtruediv__', '__rxor__', '__setattr__',
'__str__', '__sub__', '__truediv__', '__xor__']

Types

type
Python is a dynamically typed language, meaning that variables do not have a type and therefore do not have to be declared. Values, on the other hand, do have a type. You can query a variable for the type of value it contains:
1.
2.
3.
4.
5.
6.
7.
8.
9.
>>> a = 3
>>> print type(a)
<
type 'int'>
>>>
a = 3.14
>>> print type(a)
<
type 'float'>
>>>
a = 'hello python'
>>> print type(a)
<
type 'str'>

Python also includes, natively, data structures such as lists and dictionaries.

str

str
ASCII
UTF8
Unicode
encode

Python supports the use of two different types of strings: ASCII strings and Unicode strings. ASCII strings are delimited by '...', "..." or by '..' or """...""". Triple quotes delimit multiline strings. Unicode strings start with a u followed by the string containing Unicode characters. A Unicode string can be converted into an ASCII string by choosing an encoding for example:

1.
2.
3.
>>> a = 'this is an ASCII string'
>>> b = u'This is a Unicode string'
>>> a = b.encode('utf8')

After executing these three commands, the resulting a is an ASCII string storing UTF8 encoded characters. By design, web2py uses UTF8 encoded strings internally.

It is also possible to write variables into strings in various ways:

1.
2.
3.
4.
5.
6.
>>> print 'number is ' + str(3)
number is 3
>>> print 'number is %s' % (3)
number is 3
>>> print 'number is %(number)s' % dict(number=3)
number is 3

The last notation is more explicit and less error prone, and is to be preferred.

Many Python objects, for example numbers, can be serialized into strings using str or repr. These two commands are very similar but produce slightly different output. For example:

1.
2.
3.
4.
>>> for i in [3, 'hello']:
print str(i), repr(i)
3 3
hello 'hello'

For user-defined classes, str and repr can be defined/redefined using the special operators __str__ and __repr__. These are briefly described later on; for more, refer to the official Python documentation[pydocs] . repr always has a default value.

Another important characteristic of a Python string is that, like a list, it is an iterable object.

1.
2.
3.
4.
5.
6.
7.
>>> for i in 'hello':
print i
h
e
l
l
o

list

list

The main methods of a Python list are append, insert, and delete:

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
>>> a = [1, 2, 3]
>>>
print type(a)
<
type 'list'>
>>>
a.append(8)
>>>
a.insert(2, 7)
>>>
del a[0]
>>>
print a
[2, 7, 3, 8]
>>>
print len(a)
4

Lists can be sliced:

1.
2.
3.
4.
5.
6.
>>> print a[:3]
[
2, 7, 3]
>>>
print a[1:]
[
7, 3, 8]
>>>
print a[-2:]
[
3, 8]

and concatenated:

1.
2.
3.
4.
>>> a = [2, 3]
>>>
b = [5, 6]
>>>
print a + b
[2, 3, 5, 6]

A list is iterable; you can loop over it:

1.
2.
3.
4.
5.
6.
>>> a = [1, 2, 3]
>>>
for i in a:
print i
1
2
3

The elements of a list do not have to be of the same type; they can be any type of Python object.

There is a very common situation for which a list comprehension can be used. Consider the following code:

1.
2.
3.
4.
5.
6.
7.
>>> a = [1,2,3,4,5]
>>>
b = []
>>>
for x in a:
if x % 2 == 0:
b.append(x * 3)
>>>
b
[6, 12]

This code clearly processes a list of items, selects and modifies a subset of the input list, and creates a new result list, and this code can be entirely replaced with the following list comprehension:

1.
2.
3.
4.
>>> a = [1,2,3,4,5]
>>>
b = [x * 3 for x in a if x % 2 == 0]
>>>
b
[6, 12]

tuple

tuple

A tuple is like a list, but its size and elements are immutable, while in a list they are mutable. If a tuple element is an object, the object attributes are mutable. A tuple is delimited by round brackets.

1.
>>> a = (1, 2, 3)

So while this works for a list:

1.
2.
3.
4.
>>> a = [1, 2, 3]
>>>
a[1] = 5
>>> print a
[1, 5, 3]

the element assignment does not work for a tuple:

1.
2.
3.
4.
5.
6.
7.
>>> a = (1, 2, 3)
>>>
print a[1]
2
>>> a[1] = 5
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment

A tuple, like a list, is an iterable object. Notice that a tuple consisting of a single element must include a trailing comma, as shown below:

1.
2.
3.
4.
5.
6.
>>> a = (1)
>>>
print type(a)
<
type 'int'>
>>>
a = (1,)
>>>
print type(a)
<
type 'tuple'>

Tuples are very useful for efficient packing of objects because of their immutability, and the brackets are often optional:

1.
2.
3.
4.
5.
6.
>>> a = 2, 3, 'hello'
>>> x, y, z = a
>>> print x
2
>>> print z
hello

dict

dict

A Python dict-ionary is a hash table that maps a key object to a value object. For example:

1.
2.
3.
4.
5.
6.
7.
8.
9.
>>> a = {'k':'v', 'k2':3}
>>>
a['k']
v
>>> a['k2']
3
>>> a.has_key('k')
True
>>> a.has_key('v')
False

Keys can be of any hashable type (int, string, or any object whose class implements the __hash__ method). Values can be of any type. Different keys and values in the same dictionary do not have to be of the same type. If the keys are alphanumeric characters, a dictionary can also be declared with the alternative syntax:

1.
2.
3.
4.
5.
>>> a = dict(k='v', h2=3)
>>>
a['k']
v
>>> print a
{'k':'v', 'h2':3}

Useful methods are has_key, keys, values and items:

1.
2.
3.
4.
5.
6.
7.
>>> a = dict(k='v', k2=3)
>>>
print a.keys()
[
'k', 'k2']
>>>
print a.values()
[
'v', 3]
>>>
print a.items()
[(
'k', 'v'), ('k2', 3)]

The items method produces a list of tuples, each containing a key and its associated value.

Dictionary elements and list elements can be deleted with the command del:

1.
2.
3.
4.
5.
6.
7.
8.
>>> a = [1, 2, 3]
>>>
del a[1]
>>>
print a
[1, 3]
>>>
a = dict(k='v', h2=3)
>>>
del a['h2']
>>>
print a
{'k':'v'}

Internally, Python uses the hash operator to convert objects into integers, and uses that integer to determine where to store the value.

1.
2.
>>> hash("hello world")
-
1500746465

About indentation

Python uses indentation to delimit blocks of code. A block starts with a line ending in colon, and continues for all lines that have a similar or higher indentation as the next line. For example:

1.
2.
3.
4.
5.
6.
7.
8.
>>> i = 0
>>> while i < 3:
>>>
print i
>>> i = i + 1
>>>
0
1
2

It is common to use four spaces for each level of indentation. It is a good policy not to mix tabs with spaces, which can result in (invisible) confusion.

for...in

for
In Python, you can loop over iterable objects:
1.
2.
3.
4.
5.
6.
7.
>>> a = [0, 1, 'hello', 'python']
>>>
for i in a:
print i
0
1
hello
python

One common shortcut is xrange, which generates an iterable range without storing the entire list of elements.

1.
2.
3.
4.
5.
6.
>>> for i in xrange(0, 4):
print i
0
1
2
3

This is equivalent to the C/C++/C#/Java syntax:

1.
for(int i=0; i<4; i=i+1) { print(i); }

Another useful command is enumerate, which counts while looping:

1.
2.
3.
4.
5.
6.
7.
>>> a = [0, 1, 'hello', 'python']
>>>
for i, j in enumerate(a):
print i, j
0 0
1 1
2
hello
3 python

There is also a keyword range(a, b, c) that returns a list of integers starting with the value a, incrementing by c, and ending with the last value smaller than b, a defaults to 0 and c defaults to 1. xrange is similar but does not actually generate the list, only an iterator over the list; thus it is better for looping.

You can jump out of a loop using break

1.
2.
3.
4.
>>> for i in [1, 2, 3]:
print i
break
1

You can jump to the next loop iteration without executing the entire code block with continue

1.
2.
3.
4.
5.
6.
7.
>>> for i in [1, 2, 3]:
print i
continue
print
'test'
1
2
3

while

while
The while loop in Python works much as it does in many other programming languages, by looping an indefinite number of times and testing a condition before each iteration. If the condition is False, the loop ends.
1.
2.
3.
4.
5.
>>> i = 0
>>> while i < 10:
i = i + 1
>>> print i
10

There is no loop...until construct in Python.

if...elif...else

if
elif
else
The use of conditionals in Python is intuitive:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
>>> for i in range(3):
>>>
if i == 0:
>>>
print 'zero'
>>> elif i == 1:
>>>
print 'one'
>>> else:
>>>
print 'other'
zero
one
other

"elif" means "else if". Both elif and else clauses are optional. There can be more than one elif but only one else statement. Complex conditions can be created using the not, and and or operators.

1.
2.
3.
>>> for i in range(3):
>>>
if i == 0 or (i == 1 and i + 1 == 2):
>>>
print '0 or 1'

try...except...else...finally

try
except
finally
Exception
Python can throw - pardon, raise - Exceptions:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
>>> try:
>>>
a = 1 / 0
>>> except Exception, e:
>>>
print 'oops: %s' % e
>>> else:
>>>
print 'no problem here'
>>> finally:
>>>
print 'done'
oops: integer division or modulo by zero
done

If the exception is raised, it is caught by the except clause, which is executed, while the else clause is not. If no exception is raised, the except clause is not executed, but the else one is. The finally clause is always executed.

There can be multiple except clauses for different possible exceptions:

1.
2.
3.
4.
5.
6.
7.
>>> try:
>>>
raise SyntaxError
>>> except ValueError:
>>>
print 'value error'
>>> except SyntaxError:
>>>
print 'syntax error'
syntax error

The else and finally clauses are optional.

Here is a list of built-in Python exceptions + HTTP (defined by web2py)

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
BaseException
+-- HTTP (defined by web2py)
+--
SystemExit
+-- KeyboardInterrupt
+-- Exception
+-- GeneratorExit
+-- StopIteration
+-- StandardError
| +-- ArithmeticError
| | +-- FloatingPointError
| | +-- OverflowError
| | +-- ZeroDivisionError
| +-- AssertionError
| +-- AttributeError
| +-- EnvironmentError
| | +-- IOError
| | +-- OSError
| | +-- WindowsError (Windows)
| | +--
VMSError (VMS)
| +--
EOFError
| +-- ImportError
| +-- LookupError
| | +-- IndexError
| | +-- KeyError
| +-- MemoryError
| +-- NameError
| | +-- UnboundLocalError
| +-- ReferenceError
| +-- RuntimeError
| | +-- NotImplementedError
| +-- SyntaxError
| | +-- IndentationError
| | +-- TabError
| +-- SystemError
| +-- TypeError
| +-- ValueError
| | +-- UnicodeError
| | +-- UnicodeDecodeError
| | +-- UnicodeEncodeError
| | +-- UnicodeTranslateError
+-- Warning
+-- DeprecationWarning
+-- PendingDeprecationWarning
+-- RuntimeWarning
+-- SyntaxWarning
+-- UserWarning
+-- FutureWarning
+-- ImportWarning
+-- UnicodeWarning

For a detailed description of each of them, refer to the official Python documentation.

web2py exposes only one new exception, called HTTP. When raised, it causes the program to return an HTTP error page (for more on this refer to Chapter 4).

Any object can be raised as an exception, but it is good practice to raise objects that extend one of the built-in exception classes.

def...return

def
return

Functions are declared using def. Here is a typical Python function:

1.
2.
3.
4.
>>> def f(a, b):
return a + b
>>> print f(4, 2)
6

There is no need (or way) to specify types of the arguments or the return type(s). In this example, a function f is defined that can take two arguments.

Functions are the first code syntax feature described in this chapter to introduce the concept of scope, or namespace. In the above example, the identifiers a and b are undefined outside of the scope of function f:

1.
2.
3.
4.
5.
6.
7.
8.
9.
>>> def f(a):
return a + 1
>>> print f(1)
2
>>> print a
Traceback (most recent call last):
File "<pyshell#22>", line 1, in <module>
print a
NameError: name 'a' is not defined

Identifiers defined outside of function scope are accessible within the function; observe how the identifier a is handled in the following code:

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
>>> a = 1
>>> def f(b):
return a + b
>>> print f(1)
2
>>> a = 2
>>> print f(1) # new value of a is used
3
>>> a = 1 # reset a
>>> def g(b):
a = 2 # creates a new local a
return a + b
>>> print g(2)
4
>>> print a # global a is unchanged
1

If a is modified, subsequent function calls will use the new value of the global a because the function definition binds the storage location of the identifier a, not the value of a itself at the time of function declaration; however, if a is assigned-to inside function g, the global a is unaffected because the new local a hides the global value. The external-scope reference can be used in the creation of closures:

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
>>> def f(x):
def g(y):
return x * y
return g
>>> doubler = f(2) # doubler is a new function
>>> tripler = f(3) # tripler is a new function
>>> quadrupler = f(4) # quadrupler is a new function
>>> print doubler(5)
10
>>> print tripler(5)
15
>>> print quadrupler(5)
20

Function f creates new functions; and note that the scope of the name g is entirely internal to f. Closures are extremely powerful.

Function arguments can have default values, and can return multiple results:

1.
2.
3.
4.
5.
6.
7.
>>> def f(a, b=2):
return a + b, a - b
>>> x, y = f(5)
>>>
print x
7
>>> print y
3

Function arguments can be passed explicitly by name, and this means that the order of arguments specified in the caller can be different than the order of arguments with which the function was defined:

1.
2.
3.
4.
5.
6.
7.
>>> def f(a, b=2):
return a + b, a - b
>>> x, y = f(b=5, a=2)
>>>
print x
7
>>> print y
-3

Functions can also take a runtime-variable number of arguments:

1.
2.
3.
4.
5.
6.
7.
>>> def f(*a, **b):
return a, b
>>> x, y = f(3, 'hello', c=4, test='world')
>>>
print x
(3, 'hello')
>>>
print y
{'c':4, 'test':'world'}

Here arguments not passed by name (3, 'hello') are stored in the tuple a, and arguments passed by name (c and test) are stored in the dictionary b.

In the opposite case, a list or tuple can be passed to a function that requires individual positional arguments by unpacking them:

1.
2.
3.
4.
5.
>>> def f(a, b):
return a + b
>>> c = (1, 2)
>>>
print f(*c)
3

and a dictionary can be unpacked to deliver keyword arguments:

1.
2.
3.
4.
5.
>>> def f(a, b):
return a + b
>>> c = {'a':1, 'b':2}
>>>
print f(**c)
3

lambda

lambda

lambda provides a way to create a very short unnamed function very easily:

1.
2.
3.
>>> a = lambda b: b + 2
>>> print a(3)
5

The expression "lambda [a]:[b]" literally reads as "a function with arguments [a] that returns [b]". The lambda expression is itself unnamed, but the function acquires a name by being assigned to identifier a. The scoping rules for def apply to lambda equally, and in fact the code above, with respect to a, is identical to the function declaration using def:

1.
2.
3.
4.
>>> def a(b):
return b + 2
>>> print a(3)
5

The only benefit of lambda is brevity; however, brevity can be very convenient in certain situations. Consider a function called map that applies a function to all items in a list, creating a new list:

1.
2.
3.
>>> a = [1, 7, 2, 5, 4, 8]
>>>
map(lambda x: x + 2, a)
[
3, 9, 4, 7, 6, 10]
This code would have doubled in size had def been used instead of lambda. The main drawback of lambda is that (in the Python implementation) the syntax allows only for a single expression; however, for longer functions, def can be used and the extra cost of providing a function name decreases as the length of the function grows. Just like def, lambda can be used to curry functions: new functions can be created by wrapping existing functions such that the new function carries a different set of arguments:
1.
2.
3.
4.
>>> def f(a, b): return a + b
>>> g = lambda a: f(a, 3)
>>>
g(2)
5

There are many situations where currying is useful, but one of those is directly useful in web2py: caching. Suppose you have an expensive function that checks whether its argument is prime:

1.
2.
3.
4.
5.
def isprime(number):
for p in range(2, number):
if (number % p) == 0:
return False
return True

This function is obviously time consuming.

Suppose you have a caching function cache.ram that takes three arguments: a key, a function and a number of seconds.

1.
value = cache.ram('key', f, 60)

The first time it is called, it calls the function f(), stores the output in a dictionary in memory (let's say "d"), and returns it so that value is:

1.
value = d['key']=f()

The second time it is called, if the key is in the dictionary and not older than the number of seconds specified (60), it returns the corresponding value without performing the function call.

1.
value = d['key']

How would you cache the output of the function isprime for any input? Here is how:

1.
2.
3.
4.
5.
6.
>>> number = 7
>>> seconds = 60
>>> print cache.ram(str(number), lambda: isprime(number), seconds)
True
>>> print cache.ram(str(number), lambda: isprime(number), seconds)
True

The output is always the same, but the first time cache.ram is called, isprime is called; the second time it is not.

Python functions, created with either def or lambda allow re-factoring existing functions in terms of a different set of arguments. cache.ram and cache.disk are web2py caching functions.

class

class

Because Python is dynamically typed, Python classes and objects may seem odd. In fact, you do not need to define the member variables (attributes) when declaring a class, and different instances of the same class can have different attributes. Attributes are generally associated with the instance, not the class (except when declared as "class attributes", which is the same as "static member variables" in C++/Java).

Here is an example:

1.
2.
3.
4.
5.
>>> class MyClass(object): pass
>>> myinstance = MyClass()
>>>
myinstance.myvariable = 3
>>> print myinstance.myvariable
3

Notice that pass is a do-nothing command. In this case it is used to define a class MyClass that contains nothing. MyClass() calls the constructor of the class (in this case the default constructor) and returns an object, an instance of the class. The (object) in the class definition indicates that our class extends the built-in object class. This is not required, but it is good practice.

Here is a more complex class:

1.
2.
3.
4.
5.
6.
7.
8.
9.
>>> class MyClass(object):
>>>
z = 2
>>> def __init__(self, a, b):
>>>
self.x = a, self.y = b
>>> def add(self):
>>>
return self.x + self.y + self.z
>>> myinstance = MyClass(3, 4)
>>>
print myinstance.add()
9

Functions declared inside the class are methods. Some methods have special reserved names. For example, __init__ is the constructor. All variables are local variables of the method except variables declared outside methods. For example, z is a class variable, equivalent to a C++ static member variable that holds the same value for all instances of the class.

Notice that __init__ takes 3 arguments and add takes one, and yet we call them with 2 and 0 arguments respectively. The first argument represents, by convention, the local name used inside the method to refer to the current object. Here we use self to refer to the current object, but we could have used any other name. self plays the same role as *this in C++ or this in Java, but self is not a reserved keyword.

This syntax is necessary to avoid ambiguity when declaring nested classes, such as a class that is local to a method inside another class.

Special attributes, methods and operators

Class attributes, methods, and operators starting with a double underscore are usually intended to be private (i.e. to be used internally but not exposed outside the class) although this is a convention that is not enforced by the interpreter.

Some of them are reserved keywords and have a special meaning.

Here, as an example, are three of them:

They can be used, for example, to create a container object that acts like a list:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
>>> class MyList(object):
>>>
def __init__(self, *a): self.a = list(a)
>>>
def __len__(self): return len(self.a)
>>>
def __getitem__(self, i): return self.a[i]
>>>
def __setitem__(self, i, j): self.a[i] = j
>>> b = MyList(3, 4, 5)
>>>
print b[1]
4
>>> b.a[1] = 7
>>> print b.a
[3, 7, 5]

Other special operators include __getattr__ and __setattr__, which define the get and set attributes for the class, and __sum__ and __sub__, which overload arithmetic operators. For the use of these operators we refer the reader to more advanced books on this topic. We have already mentioned the special operators __str__ and __repr__.

File input/output

file.read
file.write

In Python you can open and write in a file with:

1.
2.
3.
>>> file = open('myfile.txt', 'w')
>>>
file.write('hello world')
>>>
file.close()

Similarly, you can read back from the file with:

1.
2.
3.
>>> file = open('myfile.txt', 'r')
>>>
print file.read()
hello world

Alternatively, you can read in binary mode with "rb", write in binary mode with "wb", and open the file in append mode "a", using standard C notation.

The read command takes an optional argument, which is the number of bytes. You can also jump to any location in a file using seek.

file.seek

You can read back from the file with read

1.
2.
3.
>>> print file.seek(6)
>>>
print file.read()
world

and you can close the file with:

1.
>>> file.close()
In the standard distribution of Python, which is known as CPython, variables are reference-counted, including those holding file handles, so CPython knows that when the reference count of an open file handle decreases to zero, the file may be closed and the variable disposed. However, in other implementations of Python such as PyPy, garbage collection is used instead of reference counting, and this means that it is possible that there may accumulate too many open file handles at one time, resulting in an error before the gc has a chance to close and dispose of them all. Therefore it is best to explicitly close file handles when they are no longer needed. web2py provides two helper functions, read_file() and write_file() inside the gluon.fileutils namespace that encapsulate the file access and ensure that the file handles being used are properly closed.
When using web2py, you do not know where the current directory is, because it depends on how web2py is configured. The variable request.folder contains the path to the current application. Paths can be concatenated with the command os.path.join, discussed below.

exec, eval

exec
eval

Unlike Java, Python is a truly interpreted language. This means it has the ability to execute Python statements stored in strings. For example:

1.
2.
3.
>>> a = "print 'hello world'"
>>> exec(a)
'hello world'

What just happened? The function exec tells the interpreter to call itself and execute the content of the string passed as argument. It is also possible to execute the content of a string within a context defined by the symbols in a dictionary:

1.
2.
3.
4.
>>> a = "print b"
>>> c = dict(b=3)
>>>
exec(a, {}, c)
3

Here the interpreter, when executing the string a, sees the symbols defined in c (b in the example), but does not see c or a themselves. This is different than a restricted environment, since exec does not limit what the inner code can do; it just defines the set of variables visible to the code.

A related function is eval, which works very much like exec except that it expects the argument to evaluate to a value, and it returns that value.

1.
2.
3.
4.
>>> a = "3*4"
>>> b = eval(a)
>>>
print b
12

import

import
random
The real power of Python is in its library modules. They provide a large and consistent set of Application Programming Interfaces (APIs) to many system libraries (often in a way independent of the operating system).

For example, if you need to use a random number generator, you can do:

1.
2.
3.
>>> import random
>>> print random.randint(0, 9)
5

This prints a random integer between 0 and 9 (including 9), 5 in the example. The function randint is defined in the module random. It is also possible to import an object from a module into the current namespace:

1.
2.
>>> from random import randint
>>> print randint(0, 9)

or import all objects from a module into the current namespace:

1.
2.
>>> from random import *
>>>
print randint(0, 9)

or import everything in a newly defined namespace:

1.
2.
>>> import random as myrand
>>> print myrand.randint(0, 9)

In the rest of this book, we will mainly use objects defined in modules os, sys, datetime, time and cPickle.

All of the web2py objects are accessible via a module called gluon, and that is the subject of later chapters. Internally, web2py uses many Python modules (for example thread), but you rarely need to access them directly.
In the following subsections we consider those modules that are most useful.

os

os
os.path.join
os.unlink

This module provides an interface to the operating system API. For example:

1.
2.
3.
>>> import os
>>> os.chdir('..')
>>>
os.unlink('filename_to_be_deleted')
Some of the os functions, such as chdir, MUST NOT be used in web2py because they are not thread-safe.
os.path.join is very useful; it allows the concatenation of paths in an OS-independent way:
1.
2.
3.
4.
>>> import os
>>> a = os.path.join('path', 'sub_path')
>>>
print a
path/sub_path

System environment variables can be accessed via:

1.
>>> print os.environ

which is a read-only dictionary.

sys

sys
sys.path

The sys module contains many variables and functions, but the one we use the most is sys.path. It contains a list of paths where Python searches for modules. When we try to import a module, Python looks for it in all the folders listed in sys.path. If you install additional modules in some location and want Python to find them, you need to append the path to that location to sys.path.

1.
2.
>>> import sys
>>> sys.path.append('path/to/my/modules')

When running web2py, Python stays resident in memory, and there is only one sys.path, while there are many threads servicing the HTTP requests. To avoid a memory leak, it is best to check if a path is already present before appending:

1.
2.
3.
>>> path = 'path/to/my/modules'
>>> if not path in sys.path:
sys.path.append(path)

datetime

date
datetime
time

The use of the datetime module is best illustrated by some examples:

1.
2.
3.
4.
5.
>>> import datetime
>>> print datetime.datetime.today()
2008-07-04 14:03:90
>>> print datetime.date.today()
2008-07-04

Occasionally you may need to time-stamp data based on the UTC time as opposed to local time. In this case you can use the following function:

1.
2.
3.
>>> import datetime
>>> print datetime.datetime.utcnow()
2008-07-04 14:03:90

The datetime module contains various classes: date, datetime, time and timedelta. The difference between two date or two datetime or two time objects is a timedelta:

1.
2.
3.
4.
5.
>>> a = datetime.datetime(2008, 1, 1, 20, 30)
>>>
b = datetime.datetime(2008, 1, 2, 20, 30)
>>>
c = b - a
>>> print c.days
1

In web2py, date and datetime are used to store the corresponding SQL types when passed to or returned from the database.

time

time

The time module differs from date and datetime because it represents time as seconds from the epoch (beginning of 1970).

1.
2.
3.
>>> import time
>>> t = time.time()
1215138737.571

Refer to the Python documentation for conversion functions between time in seconds and time as a datetime.

cPickle

cPickle

This is a very powerful module. It provides functions that can serialize almost any Python object, including self-referential objects. For example, let's build a weird object:

1.
2.
3.
4.
>>> class MyClass(object): pass
>>> myinstance = MyClass()
>>>
myinstance.x = 'something'
>>> a = [1 ,2, {'hello':'world'}, [3, 4, [myinstance]]]

and now:

1.
2.
3.
>>> import cPickle
>>> b = cPickle.dumps(a)
>>>
c = cPickle.loads(b)

In this example, b is a string representation of a, and c is a copy of a generated by de-serializing b.

cPickle can also serialize to and de-serialize from a file:

1.
2.
>>> cPickle.dump(a, open('myfile.pickle', 'wb'))
>>>
c = cPickle.load(open('myfile.pickle', 'rb'))

Overview

Startup

Linux
Mac
Windows

web2py comes in binary packages for Windows and Mac OS X. They include the Python interpreter so you do not need to have it pre-installed. There is also a source code version that runs on Windows, Mac, Linux, and other Unix systems. The Windows and OS X binary versions include the necessary Python interpreter. The source code package assumes that Python is already installed on the computer.

web2py requires no installation. To get started, unzip the downloaded zip file for your specific operating system and execute the corresponding web2py file.

On Windows, run:

1.
web2py.exe

On OS X, run:

1.
open web2py.app

On Unix and Linux, run from source by typing:

1.
python2.5 web2py.py

To run web2py on Windows from source install first Mark Hammond's "Python for Windows extensions, then run:

1.
python2.5 web2py.py

The web2py program accepts various command line options which are discussed later.

By default, at startup, web2py displays a startup window and then displays a GUI widget that asks you to choose a one-time administrator password, the IP address of the network interface to be used for the web server, and a port number from which to serve requests. By default, web2py runs its web server on 127.0.0.1:8000 (port 8000 on localhost), but you can run it on any available IP address and port. You can query the IP address of your network interface by opening a command line and typing ipconfig on Windows or ifconfig on OS X and Linux. From now on we assume web2py is running on localhost (127.0.0.1:8000). Use 0.0.0.0:80 to run web2py publicly on any of your network interfaces.

image

If you do not provide an administrator password, the administration interface is disabled. This is a security measure to prevent publicly exposing the admin interface.

The administrative interface, admin, is only accessible from localhost unless you run web2py behind Apache with mod_proxy. If admin detects a proxy, the session cookie is set to secure and admin login does not work unless the communication between the client and the proxy goes over HTTPS; this is a security measure. All communications between the client and admin must always be local or encrypted; otherwise an attacker would be able to perform a man-in-the middle attack or a replay attack and execute arbitrary code on the server.

After the administration password has been set, web2py starts up the web browser at the page:

1.
http://127.0.0.1:8000/

If the computer does not have a default browser, open a web browser and enter the URL.

image

Clicking on "administrative interface" takes you to the login page for the administration interface.

image

The administrator password is the password you chose at startup. Notice that there is only one administrator, and therefore only one administrator password. For security reasons, the developer is asked to choose a new password every time web2py starts unless the <recycle> option is specified. This is distinct from the authentication mechanism in web2py applications.

After the administrator logs into web2py, the browser is redirected to the "site" page.

image

This page lists all installed web2py applications and allows the administrator to manage them. web2py comes with three applications:

admin
examples
welcome
scaffolding

appliances
Ready-to-use web2py applications are referred to as web2py appliances. You can download many freely available appliances from [appliances] . web2py users are encouraged to submit new appliances, either in open-source or closed-source (compiled and packed) form.

From the admin application's site page, you can perform the following operations:

When you create a new application using admin, it starts as a clone of the "welcome" scaffolding app with a "models/db.py" that creates a SQLite database, connects to it, instantiates Auth, Crud and Service, configures them. It also provides a "controller/default.py" which exposes actions "index", "download", "user" for user management, and "call" for services. In the following, we assume that these files have been removed; we will be creating apps from scratch.
web2py also comes with a wizard, described later in this chapter, that can write an alternate scaffolding code for you based on layouts and plugins available on the web and based on high level description of the models.

Say hello

index

Here, as an example, we create a simple web app that displays the message "Hello from MyApp" to the user. We will call this application "myapp". We will also add a counter that counts how many times the same user visits the page.

You can create a new application simply by typing its name in the form on the top right of the site page in admin.

image

After you press [create], the application is created as a copy of the built-in welcome application.

image

To run the new application, visit:

1.
http://127.0.0.1:8000/myapp

Now you have a copy of the welcome application.

To edit an application, click on the design button for the newly created application.

The Edit page tells you what is inside the application. Every web2py application consists of certain files, most of which fall into one of isx categories:

Everything is neatly organized following the Model-View-Controller design pattern. Each section in the edit page corresponds to a subfolder in the application folder.

Notice that section headings will toggle their content. Folder names under static files are also collapsible.

Each file listed in the section corresponds to a file physically located in the subfolder. Any operation performed on a file via the admin interface (create, edit, delete) can be performed directly from the shell using your favorite editor.
The application contains other types of files (database, session files, error files, etc.), but they are not listed on the edit page because they are not created or modified by the administrator; they are created and modified by the application itself.

The controllers contain the logic and workflow of the application. Every URL gets mapped into a call to one of the functions in the controllers (actions). There are two default controllers: "appadmin.py" and "default.py". appadmin provides the database administrative interface; we do not need it now. "default.py" is the controller that you need to edit, the one that is called by default when no controller is specified in the URL. Edit the "index" function as follows:

1.
2.
def index():
return "Hello from MyApp"

Here is what the online editor looks like:

image

Save it and go back to the edit page. Click on the index link to visit the newly created page.

When you visit the URL

1.
http://127.0.0.1:8000/myapp/default/index

the index action in the default controller of the myapp application is called. It returns a string that the browser displays for us. It should look like this:

image

Now, edit the "index" function as follows:

1.
2.
def index():
return dict(message="Hello from MyApp")

Also from the edit page, edit the view "default/index.html" (the view file associated with the action) and completely replace the existing contents of that file with the following:

1.
2.
3.
4.
5.
6.
<html>
<
head></head>
<
body>
<
h1>{{=message}}</h1>
</body>
</html>

Now the action returns a dictionary defining a message. When an action returns a dictionary, web2py looks for a view with the name

1.
[controller]/[function].[extension]

and executes it. Here [extension] is the requested extension. If no extension is specified, it defaults to "html", and that is what we will assume here. Under this assumption, the view is an HTML file that embeds Python code using special {{ }} tags. In particular, in the example, the {{=message}} instructs web2py to replace the tagged code with the value of the message returned by the action. Notice that message here is not a web2py keyword but is defined in the action. So far we have not used any web2py keywords.

If web2py does not find the requested view, it uses the "generic.html" view that comes with every application.

If an extension other than "html" is specified ("json" for example), and the view file "[controller]/[function].json" is not found, web2py looks for the view "generic.json". web2py comes with generic.html, generic.json, generic.xml, and generic.rss. These generic views can be modified for each application individually, and additional views can be added easily.
Generic views are a development tool. In production every action should have its own view. In fact, by default, generic views are only enabled from localhost.
You can also specify a view with response.view = 'default/something.html'
Read more on this topic in Chapter 10.

If you go back to "EDIT" and click on index, you will now see the following HTML page:

image

For debugging purposes you can always append

1.
{{=response.toolbar()}}

to the code in a view and it will show you some useful information, including the request, response and session objects, and list all db queries with their timing.

Let's count

session
Let's now add a counter to this page that will count how many times the same visitor displays the page.

web2py automatically and transparently tracks visitors using sessions and cookies. For each new visitor, it creates a session and assigns a unique "session_id". The session is a container for variables that are stored server-side. The unique id is sent to the browser via a cookie. When the visitor requests another page from the same application, the browser sends the cookie back, it is retrieved by web2py, and the corresponding session is restored.

To use the session, modify the default controller:

1.
2.
3.
4.
5.
6.
def index():
if not session.counter:
session.counter = 1
else:
session.counter += 1
return dict(message="Hello from MyApp", counter=session.counter)

Notice that counter is not a web2py keyword but session is. We are asking web2py to check whether there is a counter variable in the session and, if not, to create one and set it to 1. If the counter is there, we ask web2py to increase the counter by 1. Finally we pass the value of the counter to the view.

A more compact way to code the same function is this:

1.
2.
3.
def index():
session.counter = (session.counter or 0) + 1
return dict(message="Hello from MyApp", counter=session.counter)

Now modify the view to add a line that displays the value of the counter:

1.
2.
3.
4.
5.
6.
7.
<html>
<
head></head>
<
body>
<
h1>{{=message}}</h1>
<h2>Number of visits: {{=counter}}</h2>
</body>
</html>

When you visit the index page again (and again) you should get the following HTML page:

image

The counter is associated with each visitor, and is incremented each time the visitor reloads the page. Different visitors see different counters.

Say my name

form
request.vars

Now create two pages (first and second), where the first page creates a form, asks the visitor's name, and redirects to the second page, which greets the visitor by name.

yUML diagram

Write the corresponding actions in the default controller:

1.
2.
3.
4.
5.
def first():
return dict()

def second():
return dict()

Then create a view "default/first.html" for the first action, and enter:

1.
2.
3.
4.
5.
6.
{{extend 'layout.html'}}
What is your name?
<form action="second">
<input name="visitor_name" />
<input type="submit" />
</form>

Finally, create a view "default/second.html" for the second action:

1.
2.
{{extend 'layout.html'}}
<h1>Hello {{=request.vars.visitor_name}}</h1>

layout
In both views we have extended the basic "layout.html" view that comes with web2py. The layout view keeps the look and feel of the two pages coherent. The layout file can be edited and replaced easily, since it mainly contains HTML code.

If you now visit the first page, type your name:

image

and submit the form, you will receive a greeting:

image

Postbacks

redirect
URL
postback

The mechanism for form submission that we used before is very common, but it is not good programming practice. All input should be validated and, in the above example, the burden of validation would fall on the second action. Thus the action that performs the validation is different from the action that generated the form. This tends to cause redundancy in the code.

A better pattern for form submission is to submit forms to the same action that generated them, in our example the "first". The "first" action should receive the variables, process them, store them server-side, and redirect the visitor to the "second" page, which retrieves the variables. This mechanism is called a postback.

yUML diagram

Modify the default controller to implement self-submission:

1.
2.
3.
4.
5.
6.
7.
8.
def first():
if request.vars.visitor_name:
session.visitor_name = request.vars.visitor_name
redirect(URL('second'))
return dict()

def second():
return dict()

Then modify the "default/first.html" view:

1.
2.
3.
4.
5.
6.
{{extend 'layout.html'}}
What is your name?
<form>
<input name="visitor_name" />
<input type="submit" />
</form>

and the "default/second.html" view needs to retrieve the data from the session instead of from the request.vars:

1.
2.
{{extend 'layout.html'}}
<h1>Hello {{=session.visitor_name or "anonymous"}}</h1>

From the point of view of the visitor, the self-submission behaves exactly the same as the previous implementation. We have not added validation yet, but it is now clear that validation should be performed by the first action.

This approach is better also because the name of the visitor stays in the session, and can be accessed by all actions and views in the applications without having to be passed around explicitly.

Note that if the "second" action is ever called before a visitor name is set, it will display "Hello anonymous" because session.visitor_name returns None. Alternatively we could have added the following code in the controller (inside the second function):

1.
2.
if not request.function=='first' and not session.visitor_name:
redirect(URL('first'))

This is a general mechanism that you can use to enforce authorization on controllers, though see Chapter 9 for a more powerful method.

FORM
INPUT
requires
IS_NOT_EMPTY
accepts

With web2py we can move one step further and ask web2py to generate the form for us, including validation. web2py provides helpers (FORM, INPUT, TEXTAREA, and SELECT/OPTION) with the same names as the equivalent HTML tags. They can be used to build forms either in the controller or in the view.

For example, here is one possible way to rewrite the first action:

1.
2.
3.
4.
5.
6.
7.
def first():
form = FORM(INPUT(_name='visitor_name', requires=IS_NOT_EMPTY()),
INPUT(_type='submit'))
if form.process().accepted:
session.visitor_name = form.vars.visitor_name
redirect(URL('second'))
return dict(form=form)

where we are saying that the FORM tag contains two INPUT tags. The attributes of the input tags are specified by the named arguments starting with underscore. The requires argument is not a tag attribute (because it does not start by underscore) but it sets a validator for the value of visitor_name.

Here is yet another better wat to create the same form:

1.
2.
3.
4.
5.
6.
def first():
form = SQLFORM.factory(Field('visitor_name', requires=IS_NOT_EMPTY()))
if form.process().accepted:
session.visitor_name = form.vars.visitor_name
redirect(URL('second'))
return dict(form=form)

The form object can be easily serialized in HTML by embedding it in the "default/first.html" view.

1.
2.
3.
{{extend 'layout.html'}}
What is your name?
{{=form}}

The form.process() method applies the validators and returns the form itself. The form.accepted variable is set to True if the form was processed and passed validation. If the self-submitted form passes validation, it stores the variables in the session and redirects as before. If the form does not pass validation, error messages are inserted into the form and shown to the user, as below:

image

In the next section we will show how forms can be generated automatically from a model.

An image blog

upload

Here, as another example, we wish to create a web application that allows the administrator to post images and give them a name, and allows the visitors of the web site to view the named images and submit comments.

As before, from the site page in admin, create a new application called images, and navigate to the edit page:

image

We start by creating a model, a representation of the persistent data in the application (the images to upload, their names, and the comments). First, you need to create/edit a model file which, for lack of imagination, we call "db.py". We assume the code below will replace any existing code in "db.py". Models and controllers must have a .py extension since they are Python code. If the extension is not provided, it is appended by web2py. Views instead have a .html extension since they mainly contain HTML code.

Edit the "db.py" file by clicking the corresponding "edit" button:

image

and enter the following:

IS_EMAIL
IS_NOT_EMPTY
IS_IN_DB
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
db = DAL("sqlite://storage.sqlite")

db.define_table('image',
Field('title', unique=True),
Field('file', 'upload'),
format = '%(title)s')

db.define_table('comment',
Field('image_id', db.image),
Field('author'),
Field('email'),
Field('body', 'text'))

db.image.title.requires = IS_NOT_IN_DB(db, db.image.title)
db.comment.image_id.requires = IS_IN_DB(db, db.image.id, '%(title)s')
db.comment.author.requires = IS_NOT_EMPTY()
db.comment.email.requires = IS_EMAIL()
db.comment.body.requires = IS_NOT_EMPTY()

db.comment.image_id.writable = db.comment.image_id.readable = False

Let's analyze this line by line.

Line 1 defines a global variable called db that represents the database connection. In this case it is a connection to a SQLite database stored in the file "applications/images/databases/storage.sqlite". In the SQLite case, if the database does not exist, it is created. You can change the name of the file, as well as the name of the global variable db, but it is convenient to give them the same name, to make it easy to remember.

Lines 3-5 define a table "image". define_table is a method of the db object. The first argument, "image", is the name of the table we are defining. The other arguments are the fields belonging to that table. This table has a field called "title", a field called "file", and a field called "id" that serves as the table primary key ("id" is not explicitly declared because all tables have an id field by default). The field "title" is a string, and the field "file" is of type "upload". "upload" is a special type of field used by the web2py Data Abstraction Layer (DAL) to store the names of uploaded files. web2py knows how to upload files (via streaming if they are large), rename them safely, and store them.

When a table is defined, web2py takes one of several possible actions:

This behavior is called "migration". In web2py migrations are automatic, but can be disabled for each table by passing migrate=False as the last argument of define_table.

Line 6 defines a format string for the table. It determines how a record should be represented as a string. Notice that the format argument can also be a function that takes a record and returns a string. For example:

1.
format=lambda row: row.title

Lines 8-12 define another table called "comment". A comment has an "author", an "email" (we intend to store the email address of the author of the comment), a "body" of type "text" (we intend to use it to store the actual comment posted by the author), and an "image_id" field of type reference that points to db.image via the "id" field.

In line 14, db.image.title represents the field "title" of table "image". The attribute requires allows you to set requirements/constraints that will be enforced by web2py forms. Here we require that the "title" is unique:

1.
IS_NOT_IN_DB(db, db.image.title)

Notice this is optional because it is set automatically given that Field('title', unique=True).

The objects representing these constraints are called validators. Multiple validators can be grouped in a list. Validators are executed in the order they appear. IS_NOT_IN_DB(a, b) is a special validator that checks that the value of a field b for a new record is not already in a.

Line 15 requires that the field "image_id" of table "comment" is in db.image.id. As far as the database is concerned, we had already declared this when we defined the table "comment". Now we are explicitly telling the model that this condition should be enforced by web2py, too, at the form processing level when a new comment is posted, so that invalid values do not propagate from input forms to the database. We also require that the "image_id" be represented by the "title", '%(title)s', of the corresponding record.

Line 20 indicates that the field "image_id" of table "comment" should not be shown in forms, writable=False and not even in readonly forms, readable=False.

The meaning of the validators in lines 15-17 should be obvious.

format
Notice that the validator
1.
db.comment.image_id.requires = IS_IN_DB(db, db.image.id, '%(title)s')

can be omitted (and would be automatic) if we specify a format for referenced table:

1.
db.define_table('image', ..., format='%(title)s')

where the format can be a string or a function that takes a record and returns a string.

appadmin
Once a model is defined, if there are no errors, web2py creates an application administration interface to manage the database. You access it via the "database administration" link in the edit page or directly:
1.
http://127.0.0.1:8000/images/appadmin

Here is a screenshot of the appadmin interface:

image

This interface is coded in the controller called "appadmin.py" and the corresponding view "appadmin.html". From now on, we will refer to this interface simply as appadmin. It allows the administrator to insert new database records, edit and delete existing records, browse tables, and perform database joins.

The first time appadmin is accessed, the model is executed and the tables are created. The web2py DAL translates Python code into SQL statements that are specific to the selected database back-end (SQLite in this example). You can see the generated SQL from the edit page by clicking on the "sql.log" link under "models". Notice that the link is not present until the tables have been created.

image

If you were to edit the model and access appadmin again, web2py would generate SQL to alter the existing tables. The generated SQL is logged into "sql.log".

Now go back to appadmin and try to insert a new image record:

image

web2py has translated the db.image.file "upload" field into an upload form for the file. When the form is submitted and an image file is uploaded, the file is renamed in a secure way that preserves the extension, it is saved with the new name under the application "uploads" folder, and the new name is stored in the db.image.file field. This process is designed to prevent directory traversal attacks.

Notice that each field type is rendered by a widget. Default widgets can be overridden.

When you click on a table name in appadmin, web2py performs a select of all records on the current table, identified by the DAL query

1.
db.image.id > 0

and renders the result.

image

You can select a different set of records by editing the SQL query and pressing [Submit].

To edit or delete a single record, click on the record id number.

Because of the IS_IN_DB validator, the reference field "image_id" is rendered by a drop-down menu. The items in the drop-down are stored as keys (db.image.id), but are represented by their db.image.title, as specified by the validator.

Validators are powerful objects that know how to represent fields, filter field values, generate errors, and format values extracted from the field.

The following figure shows what happens when you submit a form that does not pass validation:

image

The same forms that are automatically generated by appadmin can also be generated programmatically via the SQLFORM helper and embedded in user applications. These forms are CSS-friendly, and can be customized.

Every application has its own appadmin; therefore, appadmin itself can be modified without affecting other applications.

So far, the application knows how to store data, and we have seen how to access the database via appadmin. Access to appadmin is restricted to the administrator, and it is not intended as a production web interface for the application; hence the next part of this walk-through. Specifically we want to create:

This is represented schematically here:

yUML diagram

Go back to the edit page and edit the "default.py" controller, replacing its contents with the following:

select
1.
2.
3.
def index():
images = db().select(db.image.ALL, orderby=db.image.title)
return dict(images=images)

This action returns a dictionary. The keys of the items in the dictionary are interpreted as variables passed to the view associated to the action. When developing, if there is no view, the action is rendered by the "generic.html" view that is provided with every web2py application.

The index action performs a select of all fields (db.image.ALL) from table image, ordered by db.image.title. The result of the select is a Rows object containing the records. Assign it to a local variable called images returned by the action to the view. images is iterable and its elements are the selected rows. For each row the columns can be accessed as dictionaries: images[0]['title'] or equivalently as images[0].title.

If you do not write a view, the dictionary is rendered by "views/generic.html" and a call to the index action would look like this:

image

You have not created a view for this action yet, so web2py renders the set of records in plain tabular form.

Proceed to create a view for the index action. Return to admin, edit "default/index.html" and replace its content with the following:

1.
2.
3.
4.
5.
6.
7.
{{extend 'layout.html'}}
<h1>Current Images</h1>
<ul>
{{for image in images:}}
{{=LI(A(image.title, _href=URL("show", args=image.id)))}}
{{pass}}
</ul>

The first thing to notice is that a view is pure HTML with special {{...}} tags. The code embedded in {{...}} is pure Python code with one caveat: indentation is irrelevant. Blocks of code start with lines ending in colon (:) and end in lines beginning with the keyword pass. In some cases the end of a block is obvious from context and the use of pass is not required.

Lines 5-7 loop over the image rows and for each row image display:

1.
LI(A(image.title, _href=URL('show', args=image.id))

This is a <li>...</li> tag that contains an <a href="...">...</a> tag which contains the image.title. The value of the hypertext reference (href attribute) is:

1.
URL('show', args=image.id)

i.e., the URL within the same application and controller as the current request that calls the function called "show", passing a single argument to the function, args=image.id. LI, A, etc. are web2py helpers that map to the corresponding HTML tags. Their unnamed arguments are interpreted as objects to be serialized and inserted in the tag's innerHTML. Named arguments starting with an underscore (for example _href) are interpreted as tag attributes but without the underscore. For example _href is the href attribute, _class is the class attribute, etc.

As an example, the following statement:

1.
{{=LI(A('something', _href=URL('show', args=123))}}

is rendered as:

1.
<li><a href="/images/default/show/123">something</a></li>

A handful of helpers (INPUT, TEXTAREA, OPTION and SELECT) also support some special named attributes not starting with underscore (value, and requires). They are important for building custom forms and will be discussed later.

Go back to the edit page. It now indicates that "default.py exposes index". By clicking on "index", you can visit the newly created page:

1.
http://127.0.0.1:8000/images/default/index

which looks like:

image

If you click on the image name link, you are directed to:

1.
http://127.0.0.1:8000/images/default/show/1

and this results in an error, since you have not yet created an action called "show" in controller "default.py".

Let's edit the "default.py" controller and replace its content with:

SQLFORM
accepts
response.flash
request.args
response.download
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
def index():
images = db().select(db.image.ALL, orderby=db.image.title)
return dict(images=images)

def show():
image = db(db.image.id==request.args(0)).select().first()
db.comment.image_id.default = image.id
form = SQLFORM(db.comment)
if form.process().accepted:
response.flash = 'your comment is posted'
comments = db(db.comment.image_id==image.id).select()
return dict(image=image, comments=comments, form=form)

def download():
return response.download(request, db)

The controller contains two actions: "show" and "download". The "show" action selects the image with the id parsed from the request args and all comments related to the image. "show" then passes everything to the view "default/show.html".

The image id referenced by:

1.
URL('show', args=image.id)

in "default/index.html", can be accessed as: request.args(0) from the "show" action.

The "download" action expects a filename in request.args(0), builds a path to the location where that file is supposed to be, and sends it back to the client. If the file is too large, it streams the file without incurring any memory overhead.

Notice the following statements:

The "download" action is already defined in the "default.py" controller of the scaffolding application.
The "download" action does not return a dictionary, so it does not need a view. The "show" action, though, should have a view, so return to admin and create a new view called "default/show.html".

Edit this new file and replace its content with the following:

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
{{extend 'layout.html'}}
<h1>Image: {{=image.title}}</h1>
<center>
<img width="200px"
src="{{=URL('download', args=image.file)}}" />
</center>
{{if len(comments):}}
<h2>Comments</h2><br /><p>
{{for comment in comments:}}
<p>{{=comment.author}} says <i>{{=comment.body}}</i></p>
{{pass}}</p>
{{else:}}
<h2>No comments posted yet</h2>
{{pass}}
<h2>Post a comment</h2>
{{=form}}

This view displays the image.file by calling the "download" action inside an <img ... /> tag. If there are comments, it loops over them and displays each one.

Here is how everything will appear to a visitor.

image

When a visitor submits a comment via this page, the comment is stored in the database and appended at the bottom of the page.

Adding CRUD

web2py also provides a CRUD (Create/Read/Update/Delete) API that simplifies forms even more. To use CRUD it is necessary to define it somewhere, such as in file "db.py":

1.
2.
from gluon.tools import Crud
crud = Crud(db)
These two lines are already in the scaffolding application.
The crud object provides high-level methods, for example:
1.
form = crud.create(table)

that can be used to replace the programming pattern:

1.
2.
3.
4.
form = SQLFORM(table)
if form.process().accepted:
session.flash = '...'
redirect('...')

Here, we rewrite the previous "show" action using crud and making some more improvements:

1.
2.
3.
4.
5.
6.
7.
8.
def show():
image = db.image(request.args(0)) or redirect(URL('index'))
db.comment.image_id.default = image.id
form = crud.create(db.comment,
message='your comment is posted',
next=URL(args=image.id))
comments = db(db.comment.image_id==image.id).select()
return dict(image=image, comments=comments, form=form)

First of all notice we have used the syntax

1.
db.image(request.args(0)) or redirect(...)

to fetch the required record. Since `table(id) returns None if the record is not found, we can use or redirect(...) in this case in one line.

The next argument of crud.create is the URL to redirect to after the form is accepted. The message argument is the one to be displayed upon acceptance. You can read more about CRUD in Chapter 7.

Adding Authentication

The web2py API for Role-Based Access Control is quite sophisticated, but for now we will limit ourselves to restricting access to the show action to authenticated users, deferring a more detailed discussion to Chapter 9.

To limit access to authenticated users, we need to complete three steps. In a model, for example "db.py", we need to add:

1.
2.
3.
from gluon.tools import Auth
auth = Auth(db)
auth.define_tables()

In our controller, we need to add one action:

1.
2.
def user():
return dict(form=auth())

This is sufficient to enable login, register, logout, etc. pages. The default layout will also show options to the corresponding pages in the top right corner.

image

We can now decorate the functions that we want to restrict, for example:

1.
2.
3.
4.
5.
6.
7.
8.
@auth.requires_login()
def show():
image = db.image(request.args(0)) or redirect(URL('index'))
db.comment.image_id.default = image.id
form = crud.create(db.comment, next=URL(args=image.id),
message='your comment is posted')
comments = db(db.comment.image_id==image.id).select()
return dict(image=image, comments=comments, form=form)

Any attempt to access

1.
http://127.0.0.1:8000/images/default/show/[image_id]

will require login. If the user is not logged it, the user will be redirected to

1.
http://127.0.0.1:8000/images/default/user/login

image

The user function also exposes, among others, the following actions:

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
http://127.0.0.1:8000/images/default/user/logout
http://127.0.0.1:8000/images/default/user/register
http://127.0.0.1:8000/images/default/user/profile
http://127.0.0.1:8000/images/default/user/change_password
http://127.0.0.1:8000/images/default/user/request_reset_password
http://127.0.0.1:8000/images/default/user/retrieve_username
http://127.0.0.1:8000/images/default/user/retrieve_password
http://127.0.0.1:8000/images/default/user/verify_email
http://127.0.0.1:8000/images/default/user/impersonate
http://127.0.0.1:8000/images/default/user/not_authorized

Now, a first-time user needs to register in order to be able to log in and read or post comments.

Both the auth object and the user function are already defined in the scaffolding application. The auth object is highly customizable and can deal with email verification, registration approvals, CAPTCHA, and alternate login methods via plugins.

Adding grids

We can improve this further using the SQLFORM.grid and SQLFORM.smartgrid gadgets to create a management interface for our application:

1.
2.
3.
4.
@auth.requires_membership('manager')
def manage():
grid = SQLFORM.smartgrid(db.image)
return dict(grid=grid)

with associated "views/default/manage.html"

{{extend 'layout.html'}}
<h2>Management Interface</h2>
{{=grid}}

Using appadmin create a group "manager" and make some users members of the group. They will not be able to access

http://127.0.0.1:8000/images/default/manage

and browse, search:

image

create, update and delete images and their comments:

image

Configuring the layout

You can configure the default layout by editing "views/layout.html" but you can also configure it without editing the HTML. In fact, the "static/base.css" stylesheet is well documented and described in Chapter 5. You can change color, columns, size, borders and background without editing the HTML. If you want to edit the menu, the title or the subtitle, you can do so in any model file. The scaffolding app, sets default values of these parameters in the file "models/menu.py":

1.
2.
3.
4.
5.
6.
response.title = request.application
response.subtitle = T('customize me!')
response.meta.author = 'you'
response.meta.description = 'describe your app'
response.meta.keywords = 'bla bla bla'
response.menu = [ [ 'Index', False, URL('index') ] ]

A wiki

wiki
RSS
Ajax
XMLRPC
In this section, we build a wiki, from scratch and without using the extended functionality provided by plugin_wiki which is described in chapter 12. The visitor will be able to create pages, search them (by title), and edit them. The visitor will also be able to post comments (exactly as in the previous applications), and also post documents (as attachments to the pages) and link them from the pages. As a convention, we adopt the Markmin syntax for our wiki syntax. We will also implement a search page with Ajax, an RSS feed for the pages, and a handler to search the pages via XML-RPC[xmlrpc] .

The following diagram lists the actions that we need to implement and the links we intend to build among them.

yUML diagram

Start by creating a new scaffolding app, naming it "mywiki".

The model must contain three tables: page, comment, and document. Both comment and document reference page because they belong to page. A document contains a file field of type upload as in the previous images application.

Here is the complete model:

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
db = DAL('sqlite://storage.sqlite')

from gluon.tools import *
auth = Auth(db)
auth.define_tables()
crud = Crud(db)

db.define_table('page',
Field('title'),
Field('body', 'text'),
Field('created_on', 'datetime', default=request.now),
Field('created_by', db.auth_user, default=auth.user_id),
format='%(title)s')

db.define_table('comment',
Field('page_id', db.page),
Field('body', 'text'),
Field('created_on', 'datetime', default=request.now),
Field('created_by', db.auth_user, default=auth.user_id))

db.define_table('document',
Field('page_id', db.page),
Field('name'),
Field('file', 'upload'),
Field('created_on', 'datetime', default=request.now),
Field('created_by', db.auth_user, default=auth.user_id),
format='%(name)s')

db.page.title.requires = IS_NOT_IN_DB(db, 'page.title')
db.page.body.requires = IS_NOT_EMPTY()
db.page.created_by.readable = db.page.created_by.writable = False
db.page.created_on.readable = db.page.created_on.writable = False

db.comment.body.requires = IS_NOT_EMPTY()
db.comment.page_id.readable = db.comment.page_id.writable = False
db.comment.created_by.readable = db.comment.created_by.writable = False
db.comment.created_on.readable = db.comment.created_on.writable = False

db.document.name.requires = IS_NOT_IN_DB(db, 'document.name')
db.document.page_id.readable = db.document.page_id.writable = False
db.document.created_by.readable = db.document.created_by.writable = False
db.document.created_on.readable = db.document.created_on.writable = False

Edit the controller "default.py" and create the following actions:

Here is the "default.py" controller:

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
50.
51.
52.
53.
54.
55.
56.
57.
58.
59.
def index():
""" this controller returns a dictionary rendered by the view
it lists all wiki pages
>>> index().has_key('pages')
True
"""
pages = db().select(db.page.id,db.page.title,orderby=db.page.title)
return dict(pages=pages)

@auth.requires_login()
def create():
"creates a new empty wiki page"
form = crud.create(db.page, next=URL('index'))
return dict(form=form)

def show():
"shows a wiki page"
this_page = db.page(request.args(0)) or redirect(URL('index'))
db.comment.page_id.default = this_page.id
form = crud.create(db.comment) if auth.user else None
pagecomments = db(db.comment.page_id==this_page.id).select()
return dict(page=this_page, comments=pagecomments, form=form)

@auth.requires_login()
def edit():
"edit an existing wiki page"
this_page = db.page(request.args(0)) or redirect(URL('index'))
form = crud.update(db.page, this_page,
next=URL('show',args=request.args))
return dict(form=form)

@auth.requires_login()
def documents():
"browser, edit all documents attached to a certain page"
page = db.page(request.args(0)) or redirect(URL('index'))
db.document.page_id.default = page.id
db.document.page_id.writable = False
grid = SQLFORM.grid(db.document.page_id==page.id,args=[page.id])
return dict(page=page, grid=grid)

def user():
return dict(form=auth())

def download():
"allows downloading of documents"
return response.download(request, db)

def search():
"an ajax wiki search page"
return dict(form=FORM(INPUT(_id='keyword',_name='keyword',
_onkeyup="ajax('callback', ['keyword'], 'target');")),
target_div=DIV(_id='target'))

def callback():
"an ajax callback that returns a <ul> of links to wiki pages"
query = db.page.title.contains(request.vars.keyword)
pages = db(query).select(orderby=db.page.title)
links = [A(p.title, _href=URL('show',args=p.id)) for p in pages]
return UL(*links)

Lines 2-6 provide a comment for the index action. Lines 4-5 inside the comment are interpreted by python as test code (doctest). Tests can be run via the admin interface. In this case the tests verify that the index action runs without errors.

Lines 18, 27, and 35 try to fetch a page record with the id in request.args(0).

Lines 13, 20 define and process create forms for a new page and a new comment and.

Line 28 defines and processes an update form for a wiki page.

Line 38 creates a grid object that allows to browser, add and update the comments linked to a page.

Some magic happens in line 51. The onkeyup attribute of the INPUT tag "keyword" is set. Every time the visitor releases a key, the JavaScript code inside the onkeyup attribute is executed, client-side. Here is the JavaScript code:

1.
ajax('callback', ['keyword'], 'target');
ajax is a JavaScript function defined in the file "web2py.js" which is included by the default "layout.html". It takes three parameters: the URL of the action that performs the synchronous callback, a list of the IDs of variables to be sent to the callback (["keyword"]), and the ID where the response has to be inserted ("target").

As soon as you type something in the search box and release a key, the client calls the server and sends the content of the 'keyword' field, and, when the sever responds, the response is embedded in the page itself as the innerHTML of the 'target' tag.

The 'target' tag is a DIV defined in line 52. It could have been defined in the view as well.

Here is the code for the view "default/create.html":

1.
2.
3.
{{extend 'layout.html'}}
<h1>Create new wiki page</h1>
{{=form}}

If you visit the create page, you see the following:

image

Here is the code for the view "default/index.html":

1.
2.
3.
4.
5.
6.
7.
{{extend 'layout.html'}}
<h1>Available wiki pages</h1>
[ {{=A('search', _href=URL('search'))}} ]<br />
<ul>{{for page in pages:}}
{{=LI(A(page.title, _href=URL('show', args=page.id)))}}
{{pass}}</ul>
[ {{=A('create page', _href=URL('create'))}} ]

It generates the following page:

image

Here is the code for the view "default/show.html":

markdown
MARKMIN
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
{{extend 'layout.html'}}
<h1>{{=page.title}}</h1>
[ {{=A('edit', _href=URL('edit', args=request.args))}}
| {{=A('documents', _href=URL('documents', args=request.args))}} ]<br />
{{=MARKMIN(page.body)}}
<h2>Comments</h2>
{{for comment in comments:}}
<p>{{=db.auth_user[comment.created_by].first_name}} on {{=comment.created_on}}
says <I>{{=comment.body}}</i></p>
{{pass}}
<h2>Post a comment</h2>
{{=form}}

If you wish to use markdown syntax instead of markmin syntax:

1.
from gluon.contrib.markdown import WIKI

and use WIKI instead of the MARKMIN helper. Alternatively, you can choose to accept raw HTML instead of markmin syntax. In this case you would replace:

1.
{{=MARKMIN(page.body)}}

with:

1.
{{=XML(page.body)}}

sanitize
(so that the XML does not get escaped, as by default web2py behavior).

This can be done better with:

1.
{{=XML(page.body, sanitize=True)}}

By setting sanitize=True, you tell web2py to escape unsafe XML tags such as "<script>", and thus prevent XSS vulnerabilities.

Now if, from the index page, you click on a page title, you can see the page that you have created:

image

Here is the code for the view "default/edit.html":

1.
2.
3.
4.
{{extend 'layout.html'}}
<h1>Edit wiki page</h1>
[ {{=A('show', _href=URL('show', args=request.args))}} ]<br />
{{=form}}

It generates a page that looks almost identical to the create page.

Here is the code for the view "default/documents.html":

1.
2.
3.
4.
5.
{{extend 'layout.html'}}
<h1>Documents for page: {{=page.title}}</h1>
[ {{=A('show', _href=URL('show', args=request.args))}} ]<br />
<h2>Documents</h2>
{{=grid}}

If, from the "show" page, you click on documents, you can now manage the documents attached to the page.

image

Finally here is the code for the view "default/search.html":

1.
2.
3.
4.
{{extend 'layout.html'}}
<h1>Search wiki pages</h1>
[ {{=A('listall', _href=URL('index'))}}]<br />
{{=form}}<br />{{=target_div}}

which generates the following Ajax search form:

image

You can also try to call the callback action directly by visiting, for example, the following URL:

1.
http://127.0.0.1:8000/mywiki/default/callback?keyword=wiki

If you look at the page source you see the HTML returned by the callback:

1.
<ul><li><a href="/mywiki/default/show/4">I made a Wiki</a></li></ul>

rss
Generating an RSS feed from the stored pages using web2py is easy because web2py includes gluon.contrib.rss2. Just append the following action to the default controller:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
def news():
"generates rss feed form the wiki pages"
reponse.generic_patterns = ['.rss']
pages = db().select(db.page.ALL, orderby=db.page.title)
return dict(
title = 'mywiki rss feed',
link = 'http://127.0.0.1:8000/mywiki/default/index',
description = 'mywiki news',
created_on = request.now,
items = [
dict(title = row.title,
link = URL('show', args=row.id),
description = MARKMIN(row.body).xml(),
created_on = row.created_on
) for row in pages])

and when you visit the page

1.
http://127.0.0.1:8000/mywiki/default/news.rss

you see the feed (the exact output depends on the feed reader). Notice that the dict is automatically converted to RSS, thanks to the .rss extension in the URL.

image

web2py also includes feedparser to read third-party feeds.

XMLRPC
Finally, let's add an XML-RPC handler that allows searching the wiki programmatically:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
service = Service()

@service.xmlrpc
def find_by(keyword):
"finds pages that contain keyword for XML-RPC"
return db(db.page.title.contains(keyword).select().as_list()

def call():
"exposes all registered services, including XML-RPC"
return service()

Here, the handler action simply publishes (via XML-RPC), the functions specified in the list. In this case, find_by. find_by is not an action (because it takes an argument). It queries the database with .select() and then extracts the records as a list with .response and returns the list.

Here is an example of how to access the XML-RPC handler from an external Python program.

1.
2.
3.
4.
5.
>>> import xmlrpclib
>>> server = xmlrpclib.ServerProxy(
'http://127.0.0.1:8000/mywiki/default/call/xmlrpc')
>>>
for item in server.find_by('wiki'):
print item['created_on'], item['title']

The handler can be accessed from many other programming languages that understand XML-RPC, including C, C++, C# and Java.

On date, datetime and time format

There are three different representation for each of the field types date, datetime and time:

The database representation is an internal issue and does not affect the code. Internally, at the web2py level, they are stored as datetime.date, datetime.datetime and datetime.time object respectively and they can be manipulated as such:

for page in db(db.page).select():
    print page.title, page.day, page.month, page.year

When dates are converted to strings in forms they are converted using the ISO representation

%Y-%m-%d %H:%M:%S

yet this representation in internationalized and you can use the admin stranslation page to change the format to an alternate one. For example:

%m/%b/%Y %H:%M:%S
Mind that by default English is not translated because web2py assumes the applications is already written in English. If you want internationalization to work for English you need to create the translation file (using admin) and you need declare that the application current language is something other than english, for example:
T.current_languages = ['null']

More on admin

admin

The administrative interface provides additional functionality that we briefly review here.

site

site

This page lists all installed applications. There are two forms at the bottom.

The first of them allows creating a new application by specifying its name.

Instant Press
The second form allows uploading an existing application from either a local file or a remote URL. When you upload an application, you need to specify a name for it. This can be its original name, but does not need to be. This allows installing multiple copies of the same application. You can try, for example, to upload the the Instant Press CMS created by Martin Mulone from:
1.
http://code.google.com/p/instant-press/
Web2py files are packages as .w2p files. These ones are tar gzipped files. Web2py uses the .w2p extension instead of the .tgz extension to prevent the browser from unzipping on download. They can be uncompressed manually with tar zxvf [filename] although this is never necessary.

image

Upon successful upload, web2py displays the MD5 checksum of the uploaded file. You can use it to verify that the file was not corrupted during upload. The InstantPress name will appear in the list of installed applications.

Click on the InstantPress name on admin to get it up and running.

image

You can read more about Instant Press at the following URL:

http://code.google.com/p/instant-press/

For each application the site page allows you to:

admin.py
All the functionality available from the web2py admin site page is also accessible programmatically via the API defined in the module gluon/admin.py. Simply open a python shell and import this module.

about

about
license

The about tab allows editing the description of the application and its license. These are written respectively in the ABOUT and LICENSE files in the application folder.

image

You can use MARKMIN, or gluon.contrib.markdown.WIKI syntax for these files as described in ref.[markdown2] .

edit

EDIT
You have used the edit page already in this chapter. Here we want to point out a few more functionalities of the edit page.

The image below shows the output of the test page for the welcome application.

image

The image below show the languages tab for the welcome application.

image

The image below shows how to edit a language file, in this case the "it" (Italian) language for the welcome application.

image

shell

If you click on the "shell" link under the controllers tab in edit, web2py will open a web based Python shell and will execute the models for the current application. This allows you to interactively talk to your application.

image

crontab

Also under the controllers tab in edit there is a "crontab" link. By clicking on this link you will be able to edit the web2py crontab file. This follows the same syntax as the unix crontab but does not rely on unix. In fact, it only requires web2py, and it works on Windows. It allows you to register actions that need to be executed in background at scheduled times. For more information about this, see the next chapter.

errors

errors
When programming web2py, you will inevitably make mistakes and introduce bugs. web2py helps in two ways: 1) it allows you to create tests for every function that can be run in the browser from the edit page; and 2) when an error manifests itself, a ticket is issued to the visitor and the error is logged.

Intentionally introduce an error in the images application as shown below:

1.
2.
3.
4.
def index():
images = db().select(db.image.ALL,orderby=db.image.title)
1/0
return dict(images=images)

When you access the index action, you get the following ticket:

image

Only the administrator can access the ticket:

image

The ticket shows the traceback, and the content of the file that caused the problem, and the complete state of system (variables, request, session, etc.) If the error occurs in a view, web2py shows the view converted from HTML into Python code. This allows to easily identify the logical structure of the file.

By default tickets are stored on filesystem and group by traceback. The administrative interface provides an aggregate views (type of traceback and number of occurrence) and a detailed view (all tickets are listed by ticket id). The administrator can switch between the two views.

Notice that everywhere admin shows syntax-highlighted code (for example, in error reports, web2py keywords are shown in orange). If you click on a web2py keyword, you are redirected to a documentation page about the keyword.

If you fix the divide-by-zero bug in the index action and introduce one in the index view:

1.
2.
3.
4.
5.
6.
7.
8.
9.
{{extend 'layout.html'}}

<h1>Current Images</h1>
<ul>
{{for image in images:}}
{{1/0}}
{{=LI(A(image.title, _href=URL("show", args=image.id)))}}
{{pass}}
</ul>

you get the following ticket:

image

Note that web2py has converted the view from HTML into a Python file, and the error described in the ticket refers to the generated Python code and NOT to the original view file:

image

This may seem confusing at first, but in practice it makes debugging easier, because the Python indentation highlights the logical structure of the code that you embedded in the views.

The code is shown at the bottom of the same page.

All tickets are listed under admin in the errors page for each application:

image

Mercurial

Mercurial

If you are running from source and you have the Mercurial version control libraries installed:

1.
easy_install mercurial

then the administrative interface shows one more menu item called "mercurial". It automatically creates a local Mercurial repository for the application. Pressing the "commit" button in the page will commit the current application. Mercurial creates and stores information about changes you make in your code into a hidden folder ".hg" in your app subfolder. Every app has its own ".hg" folder and its own ".hgignore" file (tells Mercurial which files to ignore).

The Mercurial web interface does allow you to browse previous commit and diff files but we do recommend you use Mercurial directly from the shell or one of the may GUI-based Mercurial clients since they are more powerful. For example they will allow you sync your app with a remote source repository:

images

You can read more about Mercurial here:

http://mercurial.selenic.com/

Admin wizard (experimental)

The admin interface includes a Wizard that can help you create a new applications. You can access the wizard from the "sites" page as shown in the image below.

image

The wizard will guide you through a series of steps involved in creating a new application:

The image below shows the second step of the process.

image

You can see a dropdown to select a layout plugin (from web2py.com/layouts), a multiple choice dropdown to check other plugins (from web2py.com/plugins) and a "login config" field where to put the Janrain "domain:key".

The other steps are pretty much self-explanatory.

The Wizard works well for what it does but it is considered an experimental feature for two reasons:

In any case the wizard is a handy tool for fast prototyping and it can be used to bootstrap a new application with an alternate layout and optional plugins.

Configuring admin

Normally there is no need to perform any configuration of admin but a few customizations are possible. After you login into admin you can edit the admin configuration file via the URL:

http://127.0.0.1:8000/admin/default/edit/admin/models/0.py
Notice that admin can be used to edit itself. In fact admin is an app as any other one.

The file "0.py" is very much self documented and if you are opening probably you already know what you are looking for. Anyway there a few customizations that are more important than others:

GAE_APPCFG = os.path.abspath(os.path.join('/usr/local/bin/appcfg.py'))
This should point to the location of the "appcfg.py" file that comes with the Google App Engine SDK. If you have the SDK you may want to change this config parameters to the correct value. It will allow you to deploy to GAE from the admin interface.
DEMO_MODE

You can also set web2py admin in demo mode:

DEMO_MODE = True
FILTER_APPS = ['welcome']
And only the apps listed in filter apps will be accessible and they will be only accessible in read-only mode.

MULTI_USER_MODE
virtual laboratory

If you are a teacher and want to expose the administrative interface to students so that students can share one administrative interface for their projects (think of a virtual lab), can do it by setting:

MULTI_USER_MODE = True
In this way students will be required to login and will only be able to access their own apps via admin. You, as first user/teacher, will be able to access them all.

Mind that this mechanism still assumes all users are trusted. All the apps created under admin run under the same credentials on the same filesystem. It is possible for an app created by a student to access the data and the source of an app created by another student.

More on appadmin

appadmin

appadmin is not intended to be exposed to the public. It is designed to help you by providing an easy access to the database. It consists of only two files: a controller "appadmin.py" and a view "appadmin.html" which are used by all actions in the controller.

The appadmin controller is relatively small and readable; it provides an example of designing a database interface.

appadmin shows which databases are available and which tables exist in each database. You can insert records and list all records for each table individually. appadmin paginates output 100 records at a time.

Once a set of records is selected, the header of the pages changes, allowing you to update or delete the selected records.

To update the records, enter an SQL assignment in the Query string field:

1.
title = 'test'

where string values must be enclosed in single quotes. Multiple fields can be separated by commas.

To delete a record, click the corresponding checkbox to confirm that you are sure.

appadmin can also perform joins if the SQL FILTER contains a SQL condition that involves two or more tables. For example, try:

1.
db.image.id == db.comment.image_id

web2py passes this along to the DAL, and it understands that the query links two tables; hence, both tables are selected with an INNER JOIN. Here is the output:

image

If you click on the number of an id field, you get an edit page for the record with the corresponding id.

If you click on the number of a reference field, you get an edit page for the referenced record.

You cannot update or delete rows selected by a join, because they involve records from multiple tables and this would be ambiguous.

In addition to its database administration capabilities, appadmin also enables you to view details about the contents of the application's cache (at /yourapp/appadmin/ccache) as well as the contents of the current request, response, and session objects (at /yourapp/appadmin/state).

appadmin replaces response.menu with its own menu, which provides links to the application's edit page in admin, the db (database administration) page, the state page, and the cache page. If your application's layout does not generate a menu using response.menu, then you will not see the appadmin menu. In that case, you can modify the appadmin.html file and add {{=MENU(response.menu)}} to display the menu.

The core

Command line options

It is possible to skip the GUI and start web2py directly from the command line by typing something like:

password
1.
python web2py.py -a 'your password' -i 127.0.0.1 -p 8000

When web2py starts, it creates a file called "parameters_8000.py" where it stores the hashed password. If you use "<ask>" as the password, web2py prompts you for it.

For additional security, you can start web2py with:

1.
python web2py.py -a '<recycle>' -i 127.0.0.1 -p 8000

In this case web2py reuses the previously stored hashed password. If no password is provided, or if the "parameters_8000.py" file is deleted, the web-based administrative interface is disabled.

PAM
On some Unix/Linux systems, if the password is
1.
<pam_user:some_user>

web2py uses the PAM password of the Operating System account of some_user to authenticate the administrator, unless blocked by the PAM configuration.

web2py normally runs with CPython (the C implementation of the Python interpreter created by Guido van Rossum), but it can also run with Jython (the Java implementation of the interpreter). The latter possibility allows the use of web2py in the context of a J2EE infrastructure. To use Jython, simply replace "python web2py.py ..." with "jython web2py.py". Details about installing Jython, zxJDBC modules required to access the databases can be found in Chapter 14.
The "web2py.py" script can take many command-line arguments specifying the maximum number of threads, enabling of SSL, etc. For a complete list type:

command line
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
50.
51.
52.
53.
54.
55.
56.
57.
58.
59.
60.
61.
62.
63.
64.
65.
66.
67.
68.
69.
70.
71.
72.
73.
74.
75.
76.
77.
78.
79.
80.
81.
82.
83.
84.
85.
86.
87.
>>> python web2py.py -h
Usage: python web2py.py

web2py Web Framework startup script. ATTENTION: unless a password
is specified (-a 'passwd'), web2py will attempt to run a GUI.
In this case command line options are ignored.

Options:
--
version show program's version number and exit
-h, --help show this help message and exit
-i IP, --ip=IP ip address of the server (127.0.0.1)
-
p PORT, --port=PORT port of server (8000)
-
a PASSWORD, --password=PASSWORD
password to be used for administration (use -a
"<recycle>" to reuse the last password))
-
c SSL_CERTIFICATE, --ssl_certificate=SSL_CERTIFICATE
file that contains ssl certificate
-k SSL_PRIVATE_KEY, --ssl_private_key=SSL_PRIVATE_KEY
file that contains ssl private key
-d PID_FILENAME, --pid_filename=PID_FILENAME
file to store the pid of the server
-l LOG_FILENAME, --log_filename=LOG_FILENAME
file to log connections
-n NUMTHREADS, --numthreads=NUMTHREADS
number of threads (deprecated)
--
minthreads=MINTHREADS
minimum number of server threads
--maxthreads=MAXTHREADS
maximum number of server threads
-s SERVER_NAME, --server_name=SERVER_NAME
server name for the web server
-q REQUEST_QUEUE_SIZE, --request_queue_size=REQUEST_QUEUE_SIZE
max number of queued requests when server unavailable
-o TIMEOUT, --timeout=TIMEOUT
timeout for individual request (10 seconds)
-
z SHUTDOWN_TIMEOUT, --shutdown_timeout=SHUTDOWN_TIMEOUT
timeout on shutdown of server (5 seconds)
-
f FOLDER, --folder=FOLDER
folder from which to run web2py
-v, --verbose increase --test verbosity
-Q, --quiet disable all output
-D DEBUGLEVEL, --debug=DEBUGLEVEL
set debug output level (0-100, 0 means all, 100 means
none; default is 30)
-
S APPNAME, --shell=APPNAME
run web2py in interactive shell or IPython (if
installed) with specified appname (if app does not
exist it will be created). APPNAME like a/c/f (c,f
optional)
-
B, --bpython run web2py in interactive shell or bpython (if
installed) with specified appname (if app does not
exist it will be created). Use combined with --shell
-P, --plain only use plain python shell; should be used with
--shell option
-M, --import_models auto import model files; default is False; should be
used with --shell option
-R PYTHON_FILE, --run=PYTHON_FILE
run PYTHON_FILE in web2py environment; should be used
with --shell option
-K SCHEDULER, --scheduler=SCHEDULER
run scheduled tasks for the specified apps
-K app1, app2, app3 requires a scheduler defined in the
models of the respective apps
-T TEST_PATH, --test=TEST_PATH
run doctests in web2py environment; TEST_PATH like
a/c/f (c,f optional)
-
W WINSERVICE, --winservice=WINSERVICE
-W install|start|stop as Windows service
-C, --cron trigger a cron run manually; usually invoked from a
system crontab
--softcron triggers the use of softcron
-N, --no-cron do not start cron automatically
-J, --cronjob identify cron-initiated command
-L CONFIG, --config=CONFIG
config file
-F PROFILER_FILENAME, --profiler=PROFILER_FILENAME
profiler filename
-t, --taskbar use web2py gui and run in taskbar (system tray)
--
nogui text-only, no GUI
-A ARGS, --args=ARGS should be followed by a list of arguments to be passed
to script, to be used with -S, -A must be the last
option
--no-banner Do not print header banner
--interfaces=INTERFACES
listen on multiple addresses:
"ip:port:cert:key;ip2:port2:cert2:key2;..." (:cert:key
optional; no spaces)

Lower-case options are used to configure the web server. The -L option tells web2py to read configuration options from a file, -W installs web2py as a windows service, while -S, -P and -M options start an interactive Python shell. The -T option finds and runs controller doctests in a web2py execution environment. For example, the following example runs doctests from all controllers in the "welcome" application:

1.
python web2py.py -vT welcome

if you run web2py as Windows Service, -W, it is not convenient to pass the configuration using command line arguments. For this reason, in the web2py folder there is a sample "options_std.py" configuration file for the internal web server:

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
import socket
import os

ip = '0.0.0.0'
port = 80
interfaces=[('0.0.0.0',80)]
#interfaces.append(('0.0.0.0',443,'ssl_private_key.pem','ssl_certificate.pem'))
password = '<recycle>' # ## <recycle> means use the previous password
pid_filename = 'httpserver.pid'
log_filename = 'httpserver.log'
profiler_filename = None
minthreads = None
maxthreads = None
server_name = socket.gethostname()
request_queue_size = 5
timeout = 30
shutdown_timeout = 5
folder = os.getcwd()
extcron = None
nocron = None

This file contains the web2py defaults. If you edit this file, you need to import it explicitly with the -L command-line option. It only works if you run web2py as a Windows Service.

Workflow

The web2py workflow is the following:

There are some caveats to keep in mind:

applications/a/models/*.py
applications/a/models/c/*.py
applications/a/models/c/f/*.py

The possible behaviors of an action are the following:

Return a string

def index(): return 'data'

Return a dictionary for a view:

def index(): return dict(key='value')

Return all local variables:

def index(): return locals()

Redirect the user to another page:

def index(): redirect(URL('other_action'))

Return an HTTP page other than "200 OK":

def index(): raise HTTP(404)

Return a helper (for example, a FORM):

def index(): return FORM(INPUT(_name='test'))
(this is mostly used for Ajax callbacks and components, see chapter 12)

When an action returns a dictionary, it may contain code generated by helpers, including forms based on database tables or forms from a factory, for example:

def index(): return dict(form=SQLFORM.factory(Field('name')).process())
(all forms generated by web2py use postbacks, see chapter 3)

Dispatching

url mapping
dispatching

web2py maps a URL of the form:

1.
http://127.0.0.1:8000/a/c/f.html

to the function f() in controller "c.py" in application "a". If f is not present, web2py defaults to the index controller function. If c is not present, web2py defaults to the "default.py" controller, and if a is not present, web2py defaults to the init application. If there is no init application, web2py tries to run the welcome application. This is shown schematically in the image below:

(The names of the default application, controller and function can be overridden in routes.py; see Default Application, Controller and Function below.

image

By default, any new request also creates a new session. In addition, a session cookie is returned to the client browser to keep track of the session.

The extension .html is optional; .html is assumed as default. The extension determines the extension of the view that renders the output of the controller function f(). It allows the same content to be served in multiple formats (html, xml, json, rss, etc.).

Functions that take arguments or start with a double underscore are not publicly exposed and can only be called by other functions.
static files
There is an exception made for URLs of the form:
1.
http://127.0.0.1:8000/a/static/filename

There is no controller called "static". web2py interprets this as a request for the file called "filename" in the subfolder "static" of the application "a".

PARTIAL CONTENT
IF_MODIFIED_SINCE
When static files are downloaded, web2py does not create a session, nor does it issue a cookie or execute the models. web2py always streams static files in chunks of 1MB, and sends PARTIAL CONTENT when the client sends a RANGE request for a subset of the file.

web2py also supports the IF_MODIFIED_SINCE protocol, and does not send the file if it is already stored in the browser's cache and if the file has not changed since that version.

When linking to an audio or video file in the static folder, if you want to force the browser to download the file instead of streaming the audio/video via a media player, add ?attachment to the URL. This tells web2py to set the Content-Disposition header of the HTTP response to "attachment". For example:

1.
<a href="/app/static/my_audio_file.mp3?attachment">Download</a>

When the above link is clicked, the browser will prompt the user to download the MP3 file rather than immediately streaming the audio. (As discussed below, you can also set HTTP response headers directly by assigning a dict of header names and their values to response.headers.)

request.application
request.controller
request.function
GET
POST
request.args
web2py maps GET/POST requests of the form:
1.
http://127.0.0.1:8000/a/c/f.html/x/y/z?p=1&q=2

to function f in controller "c.py" in application a, and it stores the URL parameters in the request variable as follows:

1.
request.args = ['x', 'y', 'z']

and:

1.
request.vars = {'p':1, 'q':2}

and:

1.
2.
3.
request.application = 'a'
request.controller = 'c'
request.function = 'f'

In the above example, both request.args[i] and request.args(i) can be used to retrieve the i-th element of the request.args, but while the former raises an exception if the list does not have such an index, the latter returns None in this case.

request.url
1.
request.url

stores the full URL of the current request (not including GET variables).

request.ajax
request.cid
1.
request.ajax

defaults False but it is True if web2py determines that the action was called by an Ajax request.

If the request is an Ajax request and it is initiated by a web2py component, the name of the component can be found in:

1.
request.cid

Components are discussed in more detail in Chapter 12.

request.get_vars
request.post_vars
request.vars
If the HTTP request is a GET, then request.env.request_method is set to "GET"; if it is a POST, request.env.request_method is set to "POST". URL query variables are stored in the request.vars Storage dictionary; they are also stored in request.get_vars (following a GET request) or request.post_vars (following a POST request).

web2py stores WSGI and web2py environment variables in request.env, for example:

1.
request.env.path_info = 'a/c/f'

and HTTP headers into environment variables, for example:

1.
request.env.http_host = '127.0.0.1:8000'
Notice that web2py validates all URLs to prevent directory traversal attacks.
URLs are only allowed to contain alphanumeric characters, underscores, and slashes; the args may contain non-consecutive dots. Spaces are replaced by underscores before validation. If the URL syntax is invalid, web2py returns an HTTP 400 error message[http:w,http:o] .

If the URL corresponds to a request for a static file, web2py simply reads and returns (streams) the requested file.

If the URL does not request a static file, web2py processes the request in the following order:

Notice that the controller and the view are executed in different copies of the same environment; therefore, the view does not see the controller, but it sees the models and it sees the variables returned by the controller action function.

If an exception (other than HTTP) is raised, web2py does the following:

If the exception is an HTTP exception, this is assumed to be the intended behavior (for example, an HTTP redirect), and all open database transactions are committed. The behavior after that is specified by the HTTP exception itself. The HTTP exception class is not a standard Python exception; it is defined by web2py.

Libraries

The web2py libraries are exposed to the user applications as global objects. For example (request, response, session, cache), classes (helpers, validators, DAL API), and functions (T and redirect).

These objects are defined in the following core files:

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
web2py.py
gluon/__init__.py gluon/highlight.py gluon/restricted.py gluon/streamer.py
gluon/admin.py gluon/html.py gluon/rewrite.py gluon/template.py
gluon/cache.py gluon/http.py gluon/rocket.py gluon/storage.py
gluon/cfs.py gluon/import_all.py gluon/sanitizer.py gluon/tools.py
gluon/compileapp.py gluon/languages.py gluon/serializers.py gluon/utils.py
gluon/contenttype.py gluon/main.py gluon/settings.py gluon/validators.py
gluon/dal.py gluon/myregex.py gluon/shell.py gluon/widget.py
gluon/decoder.py gluon/newcron.py gluon/sql.py gluon/winservice.py
gluon/fileutils.py gluon/portalocker.py gluon/sqlhtml.py gluon/xmlrpc.py
gluon/globals.py gluon/reserved_sql_keywords.py

The tar gzipped scaffolding app that ship with web2py is

1.
welcome.w2p

It is created upon installation and overwritten on upgrade.

The first time you start web2py, two new folders are created: deposit and applications. The "welcome" app is zipped into a "welcome.w2p" file to be used as a scaffolding app. The first time you start web2py, two new folders are created: deposit and applications. The "welcome" app is zipped into a "welcome.w2p" file to be used as scaffolding app. The deposit folder is used as temporary storage for installing and uninstalling applications.
web2py unit-tests are in
1.
gluon/tests/

There are handlers for connecting with various web servers:

1.
2.
3.
4.
5.
6.
cgihandler.py       # discouraged
gaehandler.py # for Google App Engine
fcgihandler.py # for FastCGI
wsgihandler.py # for WSGI
isapiwsgihandler.py # for IIS
modpythonhandler.py # deprecated

("fcgihandler" calls "gluon/contrib/gateways/fcgi.py" developed by Allan Saddi) and

anyserver.py

which is a script to interface with many different web servers, described in Chapter 13.

There are three example files:

1.
2.
3.
options_std.py
routes.example.py
router.example.py

The former is an optional configuration file that can be passed to web2py.py with the -L option. The second is an example of a URL mapping file. It is loaded automatically when renamed "routes.py". The third is an alternative syntax for URL mapping, and can also be renamed (or copied to) "routes.py".

The files

1.
2.
3.
app.yaml
index.yaml
queue.yaml

are configuration files used for deployment on the Google App Engine. You can read more about them in the Deployment Recipes chapter and on the Google Documentation pages.

There are also additional libraries, usually developed by a third party:

feedparser[feedparser] by Mark Pilgrim for reading RSS and Atom feeds:

1.
2.
gluon/contrib/__init__.py
gluon/contrib/feedparser.py

markdown2[markdown2] by Trent Mick for wiki markup:

1.
2.
gluon/contrib/markdown/__init__.py
gluon/contrib/markdown/markdown2.py

markmin markup:

1.
gluon/contrib/markmin.py

pyfpdf created my Mariano Reingart for generating PDF documents:

gluon/contrib/pyfpdf
This is not documented in this book but it is hosted and documented here:
http://code.google.com/p/pyfpdf/

pysimplesoap is a lightweight SOAP server implementation created by Mariano Reingart:

1.
gluon/contrib/pysimplesoap/

simplejsonrpc is a lightweight JSON-RPC client also created by Mariano Reingart:

jsonrpc
gluon/contrib/simplejsonrpc.py

memcache[memcache] Python API by Evan Martin:

gluon/contrib/memcache/__init__.py
gluon/contrib/memcache/memcache.py

redis_cache

redis
is a module to store cache in the redis database:
gluon/contrib/redis_cache.py

gql, a port of the DAL to the Google App Engine:

1.
gluon/contrib/gql.py

memdb, a port of the DAL on top of memcache:

1.
gluon/contrib/memdb.py

gae_memcache is an API to use memcache on the Google App Engine:

1.
gluon/contrib/gae_memcache.py

pyrtf[pyrtf] for generating Rich Text Format (RTF) documents, developed by Simon Cusack and revised by Grant Edwards:

1.
2.
3.
4.
5.
6.
7.
8.
gluon/contrib/pyrtf
gluon/contrib/pyrtf/__init__.py
gluon/contrib/pyrtf/Constants.py
gluon/contrib/pyrtf/Elements.py
gluon/contrib/pyrtf/PropertySets.py
gluon/contrib/pyrtf/README
gluon/contrib/pyrtf/Renderer.py
gluon/contrib/pyrtf/Styles.py

PyRSS2Gen[pyrss2gen] developed by Dalke Scientific Software, to generate RSS feeds:

1.
gluon/contrib/rss2.py

simplejson[simplejson] by Bob Ippolito, the standard library for parsing and writing JSON objects:

1.
2.
3.
4.
5.
gluon/contrib/simplejson/__init__.py
gluon/contrib/simplejson/decoder.py
gluon/contrib/simplejson/encoder.py
gluon/contrib/simplejson/jsonfilter.py
gluon/contrib/simplejson/scanner.py

Google Wallet [googlewallet] provides "pay now" buttons which link Google as payment processor:

1.
gluon/contrib/google_wallet.py

Stripe.com [stripe] provides a simple API for accepting credit card payments:

1.
gluon/contrib/stripe.py

AuthorizeNet [authorizenet] provides API to accept credit card payments via Authorize.net network

1.
gluon/contrib/AuthorizeNet.py

Dowcommerce [dowcommerce] is yet another credit cart processing API:

1.
gluon/contrib/DowCommerce.py

PAM[PAM] authentication API created by Chris AtLee:

1.
gluon/contrib/pam.py

A Bayesian classifier to populate the database with dummy data for testing purposes:

1.
gluon/contrib/populate.py

A file that allows interaction with the taskbar in windows, when web2py is running as a service:

1.
gluon/contrib/taskbar_widget.py

Optional login_methods and login_forms to be used for authentication:

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
gluon/contrib/login_methods/__init__.py
gluon/contrib/login_methods/basic_auth.py
gluon/contrib/login_methods/cas_auth.py
gluon/contrib/login_methods/dropbox_account.py
gluon/contrib/login_methods/email_auth.py
gluon/contrib/login_methods/extended_login_form.py
gluon/contrib/login_methods/gae_google_account.py
gluon/contrib/login_methods/ldap_auth.py
gluon/contrib/login_methods/linkedin_account.py
gluon/contrib/login_methods/loginza.py
gluon/contrib/login_methods/oauth10a_account.py
gluon/contrib/login_methods/oauth20_account.py
gluon/contrib/login_methods/openid_auth.py
gluon/contrib/login_methods/pam_auth.py
gluon/contrib/login_methods/rpx_account.py
gluon/contrib/login_methods/x509_auth.py

web2py also contains a folder with useful scripts including

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
scripts/setup-web2py-fedora.sh
scripts/setup-web2py-ubuntu.sh
scripts/setup-web2py-nginx-uwsgi-ubuntu.sh
scripts/update-web2py.sh
scripts/make_min_web2py.py
...
scripts/sessions2trash.py
scripts/sync_languages.py
scripts/tickets2db.py
scripts/tickets2email.py
...
scripts/extract_mysql_models.py
scripts/extract_pgsql_models.py
...
scripts/access.wsgi
scripts/cpdb.py

The first three are particularly useful because they attempt a complete installation and setup of a web2py production environment from scratch. Some of these are discussed in Chapter 14, but all of them contain a documentation string inside that explains their purpose and usage.

Finally web2py includes these files required to build the binary distributions.

1.
2.
3.
Makefile
setup_exe.py
setup_app.py

These are setup scripts for py2exe and py2app, respectively, and they are only required to build the binary distributions of web2py. YOU SHOULD NEVER NEED TO RUN THEM.

In summary, web2py libraries provide the following functionality:

web2py applications contain additional files, particularly third-party JavaScript libraries, such as jQuery, calendar/datepicker, EditArea and nicEdit. Their authors are acknowledged in the files themselves.

Applications

Applications developed in web2py are composed of the following parts:

Models, views, controllers, languages, and static files are accessible via the web administration [design] interface. ABOUT, README, and errors are also accessible via the administration interface through the corresponding menu items. Sessions, cache, modules and private files are accessible to the applications but not via the administration interface.

Everything is neatly organized in a clear directory structure that is replicated for every installed web2py application, although the user never needs to access the filesystem directly:

about
license
cache
controllers
databases
errors
languages
models
modules
private
session
static
tests
uploads
views
__init__.py
1.
2.
3.
__init__.py  ABOUT        LICENSE    models    views
controllers modules private tests cron
cache errors upload sessions static

"__init__.py" is an empty file which is required in order to allow Python (and web2py) to import the modules in the modules directory.

Notice that the admin application simply provides a web interface to web2py applications on the server file system. web2py applications can also be created and developed from the command-line; you don't have to use the browser admin interface. A new application can be created manually by replicating the above directory structure under ,e.g., "applications/newapp/" (or simply untar the welcome.w2p file into your new application directory). Application files can also be created and edited from the command-line without having to use the web admin interface.

API

Models, controllers, and views are executed in an environment where the following objects are already imported for us:

Global Objects:

request
response
session
cache
1.
request, response, session, cache

Internationalization:

T
internationalization
1.
T

Navigation:

redirect
HTTP
1.
redirect, HTTP

Helpers:

helpers
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
XML, URL, BEAUTIFY

A, B, BODY, BR, CENTER, CODE, COL, COLGROUP,
DIV, EM, EMBED, FIELDSET, FORM, H1, H2, H3, H4, H5, H6,
HEAD, HR, HTML, I, IFRAME, IMG, INPUT, LABEL, LEGEND,
LI, LINK, OL, UL, META, OBJECT, OPTION, P, PRE,
SCRIPT, OPTGROUP, SELECT, SPAN, STYLE,
TABLE, TAG, TD, TEXTAREA, TH, THEAD, TBODY, TFOOT,
TITLE, TR, TT, URL, XHTML, xmlescape, embed64

CAT, MARKMIN, MENU, ON

Forms and tables

SQLFORM (SQLFORM.factory, SQLFORM.grid, SQLFORM.smartgrid)

Validators:

validators
1.
2.
3.
4.
5.
6.
7.
CLEANUP, CRYPT, IS_ALPHANUMERIC, IS_DATE_IN_RANGE, IS_DATE,
IS_DATETIME_IN_RANGE, IS_DATETIME, IS_DECIMAL_IN_RANGE,
IS_EMAIL, IS_EMPTY_OR, IS_EXPR, IS_FLOAT_IN_RANGE, IS_IMAGE,
IS_IN_DB, IS_IN_SET, IS_INT_IN_RANGE, IS_IPV4, IS_LENGTH,
IS_LIST_OF, IS_LOWER, IS_MATCH, IS_EQUAL_TO, IS_NOT_EMPTY,
IS_NOT_IN_DB, IS_NULL_OR, IS_SLUG, IS_STRONG, IS_TIME,
IS_UPLOAD_FILENAME, IS_UPPER, IS_URL

Database:

DAL
1.
DAL, Field

For backward compatibility SQLDB=DAL and SQLField=Field. We encourage you to use the new syntax DAL and Field, instead of the old syntax.

Other objects and modules are defined in the libraries, but they are not automatically imported since they are not used as often.

The core API entities in the web2py execution environment are request, response, session, cache, URL, HTTP, redirect and T and are discussed below.

A few objects and functions, including Auth, Crud and Service, are defined in "gluon/tools.py" and they need to be imported as necessary:

1.
from gluon.tools import Auth, Crud, Service

Accessing the API from Python modules

Your models or controller may import python modules, and these may need to use some of the web2py API. The way for them to do it is importing them:

from gluon import *

In fact, any Python module, even if not imported by a web2py application, can import the web2py API as long as web2py is in the sys.path.

There is one caveat, though. Web2py defines some global objects (request, response, session, cache, T) that can only exist when an HTTP request is present (or is faked). Therefore, modules can access them only if they are called from an application. For this reasons they are placed into a container caller current, which is a thread local object. Here is an example.

Create a module "/myapp/modules/test.py" that contains:

from gluon import *
def ip(): return current.request.client
Now from a controller in "myapp" you can do
import test
def index():
    return "Your ip is " + test.ip()

Notice a few things:

For uniformity with normal Python behavior, by default web2py does not reload modules when changes are made. Yet this can be changed. To turn on the auto-reload feature for modules, use the track_changes function as follows (typically in a model file, before any imports):

1.
from gluon.custom_import import track_changes; track_changes(True)

From now on, every time a module is imported, the importer will check if the Python source file (.py) has changed. If it has changed, the module will be reloaded. This applies to all Python modules, even Python modules outside web2py. The mode is global and applies to all applications. Changes made to models, controllers, and views are always reloaded regardless of the mode used. To turn the mode off, use the same function with False as the argument. To know the actual tracking state, use the is_tracking_changes() function, also from gluon.custom_import.

Modules that import current can access:

and any other variable your application chooses to store in current. For example a model could do
auth = Auth(db)
from gluon import current
current.auth = auth

and now all modules imported can access

current and import create a powerful mechanism to build extensible and reusable modules for your applications.

There is one major caveat. Given from gluon import current, it is correct to use current.request and any of the other thread local objects but one should never assign them to global variables in the module, such as in
request = current.request # WRONG! DANGER!
nor one should use it assign class attributes
class MyClass:
    request = current.request # WRONG! DANGER!
This is because the thread local object must be extracted at runtime. Global variables instead are defined only once when the model is imported for the first time.

request

request
Storage
request.cookies
user_agent

The request object is an instance of the ubiquitous web2py class that is called gluon.storage.Storage, which extends the Python dict class. It is basically a dictionary, but the item values can also be accessed as attributes:

1.
request.vars

is the same as:

1.
request['vars']

Unlike a dictionary, if an attribute (or key) does not exist, it does not raise an exception. Instead, it returns None.

It is sometimes useful to create your own Storage objects. You can do so as follows:
1.
2.
3.
from gluon.storage import Storage
my_storage = Storage() # empty storage object
my_other_storage = Storage(dict(a=1, b=2)) # convert dictionary to Storage
request has the following items/attributes, some of which are also an instance of the Storage class:
1.
{{=BEAUTIFY(request.user_agent())}}

The latter includes:

their usage is discussed at the end of this Chapter.

As an example, the following call on a typical system:

1.
http://127.0.0.1:8000/examples/default/status/x/y/z?p=1&q=2

results in the following request object:

request
env
variablevalue
request.applicationexamples
request.controllerdefault
request.functionindex
request.extensionhtml
request.viewstatus
request.folderapplications/examples/
request.args['x', 'y', 'z']
request.vars<Storage {'p': 1, 'q': 2}>
request.get_vars<Storage {'p': 1, 'q': 2}>
request.post_vars<Storage {}>
request.is_localFalse
request.is_httpsFalse
request.ajaxFalse
request.cidNone
request.wsgihook
request.env.content_length0
request.env.content_type
request.env.http_accepttext/xml,text/html;
request.env.http_accept_encodinggzip, deflate
request.env.http_accept_languageen
request.env.http_cookiesession_id_examples=127.0.0.1.119725
request.env.http_host127.0.0.1:8000
request.env.http_max_forwards10
request.env.http_refererhttp://web2py.com/
request.env.http_user_agentMozilla/5.0
request.env.http_via1.1 web2py.com
request.env.http_x_forwarded_for76.224.34.5
request.env.http_x_forwarded_hostweb2py.com
request.env.http_x_forwarded_server127.0.0.1
request.env.path_info/examples/simple_examples/status
request.env.query_stringremote_addr:127.0.0.1
request.env.request_methodGET
request.env.script_name
request.env.server_name127.0.0.1
request.env.server_port8000
request.env.server_protocolHTTP/1.1
request.env.web2py_path/Users/mdipierro/web2py
request.env.web2py_versionVersion 1.99.1
request.env.web2py_runtime_gae(optional, defined only if GAE detected)
request.env.wsgi_errors<open file, mode 'w' at >
request.env.wsgi_input
request.env.wsgi_multiprocessFalse
request.env.wsgi_multithreadTrue
request.env.wsgi_run_onceFalse
request.env.wsgi_url_schemehttp
request.env.wsgi_version10
Which environment variables are actually defined depends on the web server. Here we are assuming the built-in Rocket wsgi server. The set of variables is not much different when using the Apache web server.

The request.env.http_* variables are parsed from the request HTTP header.

The request.env.web2py_* variables are not parsed from the web server environment, but are created by web2py in case your applications need to know about the web2py location and version, and whether it is running on the Google App Engine (because specific optimizations may be necessary).

Also notice the request.env.wsgi_* variables. They are specific to the wsgi adaptor.

response

response
response.body
response.cookies
response.download
response.files
response.flash
response.headers
response.meta
response.menu
response.postprocessing
response.render
response.status
response.stream
response.subtitle
response.title
response.toolbar
response.view
response.js
response.write
response.include_files
response.include_meta
response.optimize_css
response.optimize_js

response is another instance of the Storage class. It contains the following:

response.body: a StringIO object into which web2py writes the output page body. NEVER CHANGE THIS VARIABLE.

response.cookies: similar to request.cookies, but while the latter contains the cookies sent from the client to the server, the former contains cookies sent by the server to the client. The session cookie is handled automatically.

response.download(request, db): a method used to implement the controller function that allows downloading of uploaded files. request.download expects the last arg in request.args to be the encoded filename (i.e., the filename generated at upload time and stored in the upload field). It extracts the upload field name and table name as well as the original filename from the encoded filename. response.download takes two optional arguments: chunk_size sets the size in bytes for chunked streaming (defaults to 64K), and attachments determines whether the downloaded file should be treated as an attachment or not (default to True). Note, response.download is specifically for downloading files associated with db upload fields. Use response.stream (see below) for other types of file downloads and streaming. Also, note that it is not necessary to use response.download to access files uploaded to the /static folder -- static files can (and generally should) be accessed directly via URL (e.g., /app/static/files/myfile.pdf).

response.files: a list of CSS and JS files required by the page. They will automatically be linked in the head of the standard "layout.html" via the included "web2py_ajax.html". To include a new CSS or JS file, just append it to this list. It will handle duplicates. The order is significant.

response.include_files() generates html head tags to includes all response.files (used in "views/web2py_ajax.html").

response.flash: optional parameter that may be included in the views. Normally used to notify the user about something that happened.

response.headers: a dict for HTTP response headers. Web2py sets some headers by default, including "Content-Length", "Content-Type", and "X-Powered-By" (set equal to web2py). Web2py also sets the "Cache-Control", "Expires", and "Pragma" headers to prevent client-side caching, except for static file requests, for which client-side caching is enabled. The headers that web2py sets can be overwritten or removed, and new headers can be added (e.g., response.headers['Cache-Control'] = 'private').

response.menu: optional parameter that may be included in the views, normally used to pass a navigation menu tree to the view. It can be rendered by the MENU helper.

response.meta: a storage object (like a dict) that contains optional meta information like response.meta.author, .description, and/or .keywords. The content of each meta variable is automatically placed in the proper META tag by the code in "views/web2py_ajax.html", which is included by default in "views/layout.html".

response.include_meta() generates a string that includes all response.meta headers serialized (used in "views/web2py_ajax.html").

response.postprocessing: this is a list of functions, empty by default. These functions are used to filter the response object at the output of an action, before the output is rendered by the view. It can be used to implement support for other template languages.

response.render(view, vars): a method used to call the view explicitly inside the controller. view is an optional parameter which is the name of the view file, vars is a dictionary of named values passed to the view.

response.session_file: file stream containing the session.

response.session_file_name: name of the file where the session will be saved.

response.session_id: the id of the current session. It is determined automatically. NEVER CHANGE THIS VARIABLE.

response.session_id_name: the name of the session cookie for this application. NEVER CHANGE THIS VARIABLE.

response.status: the HTTP status code integer to be passed to the response. Default is 200 (OK).

response.stream(file, chunk_size, request=request): when a controller returns it, web2py streams the file content back to the client in blocks of size chunk_size. The request parameter is required to use the chunk start in the HTTP header. As noted above, response.download should be used to retrieve files stored via an upload field. response.stream can be used in other cases, such as returning a temporary file or StringIO object created by the controller. Unlike response.download, response.stream does not automatically set the Content-Disposition header, so you may need to do that manually (e.g., to specify download as an attachment, and to provide a filename). However, it does automatically set the Content-Type header (based on the filename extension).

response.subtitle: optional parameter that may be included in the views. It should contain the subtitle of the page.

response.title: optional parameter that may be included in the views. It should contain the title of the page and should be rendered by the HTML title TAG in the header.

response.toolbar: a function that allows you embed a toolbar into page form debugging purposes {{=response.toolbar()}}. The toolbar displays request, response, session variables and database access time for each query.

response._vars: this variable is accessible only in a view, not in the action. It contains the value returned by the action to the view.

response.optimize_css: if can be set to "concat,minify,inline" to concatenate, minity and inline the CSS files included by web2py.

response.optimize_js: if can be set to "concat,minify,inline" to concatenate, minity and inline the CSS files included by web2py.

response.view: the name of the view template that must render the page. This is set by default to:

1.
"%s/%s.%s" % (request.controller, request.function, request.extension)

or, if the above file cannot be located, to

1.
"generic.%s" % (request.extension)

Change the value of this variable to modify the view file associated with a particular action.

response.xmlrpc(request, methods): when a controller returns it, this function exposes the methods via XML-RPC[xmlrpc] . This function is deprecated since a better mechanism is available and described in Chapter 10.

response.write(text): a method to write text into the output page body.

response.js can contain Javascript Code. This code will be executed if and only if the response is received by a web2py component as discussed in Chapter 12.

Since response is a gluon.storage.Storage object, it can be used to store other attributes that you may want to pass to the view. While there is no technical restriction, our recommendation is to store only variables that are to be rendered by all pages in the overall layout ("layout.html").

Anyway, we strongly suggest to stick to the variables listed here:

1.
2.
3.
4.
5.
6.
7.
8.
response.title
response.subtitle
response.flash
response.menu
response.meta.author
response.meta.description
response.meta.keywords
response.meta.*

because this will make it easier for you to replace the standard "layout.html" file that comes with web2py with another layout file, one that uses the same set of variables.

Old versions of web2py user response.author instead of response.meta.author and similar for the other meta attributes.

session

session
session.connect
session.forget
session.secure
session is another instance of the Storage class. Whatever is stored into session for example:
1.
session.myvariable = "hello"

can be retrieved at a later time:

1.
a = session.myvariable

as long as the code is executed within the same session by the same user (provided the user has not deleted session cookies and the session did not expire). Because session is a Storage object, trying to access an attribute/key that has not been set does not raise an exception; it returns None instead.

The session object has three important methods. One is forget:

1.
session.forget(response)

It tells web2py not to save the session. This should be used in those controllers whose actions are called often and do not need to track user activity. session.forget() prevents the session file from being written, regardless of whether it has been modified. session.forget(response) additionally unlocks and closes the session file. You rarely need to call this method since sessions are not saved when they are not changed. However, if the page makes multiple simultaneous Ajax requests, it is a good idea for the actions called via Ajax to call session.forget(response) (assuming the session is not needed by the action). Otherwise, each Ajax action will have to wait for the previous one to complete (and unlock the session file) before proceeding, which will slow down the page loading. Notice that sessions are not locked when stored in database.

Another method is:

1.
session.secure()

which tells web2py to set the session cookie to be a secure cookie. This should be set if the app is going over https. By setting the session cookie to be secure, the server is asking the browser not to send the session cookie back to the server unless over an https connection.

The other method is connect:

1.
session.connect(request, response, db, masterapp=None)

where db is the name of an open database connection (as returned by the DAL). It tells web2py that you want to store the sessions in the database and not on the filesystem. session.connect must come after db=DAL(...), but before any other logic that requires session, for example, setting up Auth.

web2py creates a table:

1.
2.
3.
4.
5.
6.
7.
db.define_table('web2py_session',
Field('locked', 'boolean', default=False),
Field('client_ip'),
Field('created_datetime', 'datetime', default=now),
Field('modified_datetime', 'datetime'),
Field('unique_key'),
Field('session_data', 'text'))

and stores cPickled sessions in the session_data field.

The option masterapp=None, by default, tells web2py to try to retrieve an existing session for the application with name in request.application, in the running application.

If you want two or more applications to share sessions, set masterapp to the name of the master application.

You can check the state of your application at any time by printing the request, session and response system variables. One way to do it is to create a dedicated action:

1.
2.
def status():
return dict(request=request, session=session, response=response)

Separate sessions

If you are storing sessions on filesystems and you have lots of them, the file system access may become a bottle-neck. One solution is the following:

1.
session.connect(request, response, separate=True)

By setting separate=True web2py will store sessions not in the "sessions/" folder but in subfolders of the "sessions/" folder. The subfolder will be created automatically. Sessions with the same prefix will be in the same subfolder. Again, note that the above must be called before any logic that might require the session.

cache

cache
cache.ram
cache.disk
cache a global object also available in the web2py execution environment. It has two attributes:cache is callable, this allows it to be used as a decorator for caching actions and views.

The following example caches the time.ctime() function in RAM:

1.
2.
3.
4.
def cache_in_ram():
import time
t = cache.ram('time', lambda: time.ctime(), time_expire=5)
return dict(time=t, link=A('click me', _href=request.url))

The output of lambda: time.ctime() is cached in RAM for 5 seconds. The string 'time' is used as cache key.

The following example caches the time.ctime() function on disk:

1.
2.
3.
4.
def cache_on_disk():
import time
t = cache.disk('time', lambda: time.ctime(), time_expire=5)
return dict(time=t, link=A('click me', _href=request.url))

The output of lambda: time.ctime() is cached on disk (using the shelve module) for 5 seconds.

Note, the second argument to cache.ram and cache.disk must be a function or callable object. If you want to cache an existing object rather than the output of a function, you can simply return it via a lambda function:

1.
cache.ram('myobject', lambda: myobject, time_expire=60*60*24)

The next example caches the time.ctime() function to both RAM and disk:

1.
2.
3.
4.
5.
6.
def cache_in_ram_and_disk():
import time
t = cache.ram('time', lambda: cache.disk('time',
lambda: time.ctime(), time_expire=5),
time_expire=5)
return dict(time=t, link=A('click me', _href=request.url))

The output of lambda: time.ctime() is cached on disk (using the shelve module) and then in RAM for 5 seconds. web2py looks in RAM first and if not there it looks on disk. If it is not in RAM or on disk, lambda: time.ctime() is executed and the cache is updated. This technique is useful in a multiprocess environment. The two times do not have to be the same.

The following example is caching in RAM the output of the controller function (but not the view):

cache controller
1.
2.
3.
4.
5.
@cache(request.env.path_info, time_expire=5, cache_model=cache.ram)
def cache_controller_in_ram():
import time
t = time.ctime()
return dict(time=t, link=A('click me', _href=request.url))

The dictionary returned by cache_controller_in_ram is cached in RAM for 5 seconds. Note that the result of a database select cannot be cached without first being serialized. A better way is to cache the database select directly using the select method cache argument.

The following example is caching the output of the controller function on disk (but not the view):

1.
2.
3.
4.
5.
6.
@cache(request.env.path_info, time_expire=5, cache_model=cache.disk)
def cache_controller_on_disk():
import time
t = time.ctime()
return dict(time=t, link=A('click to reload',
_href=request.url))

The dictionary returned by cache_controller_on_disk is cached on disk for 5 seconds. Remember that web2py cannot cache a dictionary that contains un-pickleable objects.

It is also possible to cache the view. The trick is to render the view in the controller function, so that the controller returns a string. This is done by returning response.render(d) where d is the dictionary we intended to pass to the view. The following example caches the output of the controller function in RAM (including the rendered view):

cache view
1.
2.
3.
4.
5.
6.
@cache(request.env.path_info, time_expire=5, cache_model=cache.ram)
def cache_controller_and_view():
import time
t = time.ctime()
d = dict(time=t, link=A('click to reload', _href=request.url))
return response.render(d)
response.render(d) returns the rendered view as a string, which is now cached for 5 seconds. This is the best and fastest way of caching.

Note, time_expire is used to compare the current time with the time the requested object was last saved in the cache. It does not affect future requests. This enables time_expire to be set dynamically when an object is requested rather than being fixed when the object is saved. For example:

1.
message = cache.ram('message', lambda: 'Hello', time_expire=5)

Now, suppose the following call is made 10 seconds after the above call:

1.
message = cache.ram('message', lambda: 'Goodbye', time_expire=20)

Because time_expire is set to 20 seconds in the second call and only 10 seconds has elapsed since the message was first saved, the value "Hello" will be retrieved from the cache, and it will not be updated with "Goodbye". The time_expire value of 5 seconds in the first call has no impact on the second call.

Setting time_expire=0 (or a negative value) forces the cached item to be refreshed (because the elapsed time since the last save will always be > 0), and setting time_expire=None forces retrieval of the cached value, regardless of the time elapsed since it was saved (if time_expire is always None, the cached item will effectively never expire).

You can clear one or more cache variables with

cache clear
1.
cache.ram.clear(regex='...')

where regex is a regular expression matching all the keys you want removed from the cache. You can also clear an single item with:

1.
cache.ram(key, None)

where key is the key of the cached item.

It is also possible to define other caching mechanisms such as memcache. Memcache is available via gluon.contrib.memcache and is discussed in more details in Chapter 14.

Be careful when caching that caching is usually at the app-level not at the user level. If you need, for eaxmple, to cache user specific content, choose a key that includes the user id.

URL

URL
The URL function is one of the most important functions in web2py. It generates internal URL paths for the actions and the static files.

Here is an example:

1.
URL('f')

is mapped into

1.
/[application]/[controller]/f

Notice that the output of the URL function depends on the name of the current application, the calling controller, and other parameters. web2py supports URL mapping and reverse URL mapping. URL mapping allows you to redefine the format of external URLs. If you use the URL function to generate all the internal URLs, then additions or changes to URL mappings will prevent broken links within the web2py application.

You can pass additional parameters to the URL function, i.e., extra terms in the URL path (args) and URL query variables (vars):

1.
URL('f', args=['x', 'y'], vars=dict(z='t'))

is mapped into

1.
/[application]/[controller]/f/x/y?z=t

The args attributes are automatically parsed, decoded, and finally stored in request.args by web2py. Similarly, the vars are parsed, decoded, and then stored in request.vars. args and vars provide the basic mechanism by which web2py exchanges information with the client's browser.

If args contains only one element, there is no need to pass it in a list.

You can also use the URL function to generate URLs to actions in other controllers and other applications:

1.
URL('a', 'c', 'f', args=['x', 'y'], vars=dict(z='t'))

is mapped into

/a/c/f/x/y?z=t

It is also possible to specify application, controller and function using named arguments:

1.
URL(a='a', c='c', f='f')

If the application name a is missing the current app is assumed.

1.
URL('c', 'f')

If the controller name is missing, the current one is assumed.

1.
URL('f')

Instead of passing the name of a controller function it is also possible to pass the function itself

1.
URL(f)

For the reasons mentioned above, you should always use the URL function to generate URLs of static files for your applications. Static files are stored in the application's static subfolder (that's where they go when uploaded using the administrative interface). web2py provides a virtual 'static' controller whose job is to retrieve files from the static subfolder, determine their content-type, and stream the file to the client. The following example generates the URL for the static file "image.png":

1.
URL('static', 'image.png')

is mapped into

1.
/[application]/static/image.png

If the static file is in a subfolder within the static folder, you can include the subfolder(s) as part of the filename. For example, to generate:

/[application]/static/images/icons/arrow.png

one should use:

1.
URL('static', 'images/icons/arrow.png')

You do not need to encode/escape the args and vars arguments; this is done automatically for you.

By default, the extension corresponding to the current request (which can be found in request.extension) is appended to the function, unless request.extension is html, the default. This can be overridden by explicitly including an extension as part of the function name URL(f='name.ext') or with the extension argument:

1.
URL(..., extension='css')

The current extension can be explicitly suppressed:

1.
URL(..., extension=False)

Absolute urls

By default, URL generates relative URLs. However, you can also generate absolute URLs by specifying the scheme and host arguments (this is useful, for example, when inserting URLs in email messages):

1.
URL(..., scheme='http', host='www.mysite.com')

You can automatically include the scheme and host of the current request by simply setting the arguments to True.

1.
URL(..., scheme=True, host=True)

The URL function also accepts a port argument to specify the server port if necessary.

Digitally signed urls

digitally signed URL

When generating a URL, you have the option to digitally sign it. This will append a _signature GET variable that can be verified by the server. This can be done in two ways.

You can pass to the URL function the following arguments:

Here is an example of usage:

1.
2.
3.
4.
5.
6.
7.
8.
9.
KEY = 'mykey'

def one():
return dict(link=URL('two', vars=dict(a=123), hmac_key=KEY)

def two():
if not URL.verify(hmac_key=KEY): raise HTTP(403)
# do something
return locals()

This makes the action two accessible only via a digitally signed URL. A digitally signed URL looks like this:

'/welcome/default/two?a=123&_signature=4981bc70e13866bb60e52a09073560ae822224e9'

Note, the digital signature is verified via the URL.verify function. URL.verify also takes the hmac_key, salt, and hash_vars arguments described above, and their values must match the values that were passed to the URL function when the digital signature was created in order to verify the URL.

A second and more sophisticated but more common use of digitally signed URLs is in conjunction with Auth. This is best explained with an example:

1.
2.
3.
4.
5.
6.
7.
8.
@auth.requires_login()
def one():
return dict(link=URL('two', vars=dict(a=123), user_signature=True)

@auth.requires_signature()
def two():
# do something
return locals()

In this case the hmac_key is automatically generated and shared within the session. This allows action two to delegate any access control to action one. If the link is generated and signed, it is valid; else it is not. If the link is stolen by another user, the link will be invalid.

It is good practice to always digitally sign Ajax callbacks. If you use the web2py LOAD function, it has a user_signature argument too that can be used for this purpose:

{{=LOAD('default', 'two', vars=dict(a=123), ajax=True, user_signature=True)}}

HTTP and redirect

HTTP
redirect

web2py defines only one new exception called HTTP. This exception can be raised anywhere in a model, a controller, or a view with the command:

1.
raise HTTP(400, "my message")

It causes the control flow to jump away from the user's code, back to web2py, and return an HTTP response like:

1.
2.
3.
4.
5.
6.
7.
8.
9.
HTTP/1.1 400 BAD REQUEST
Date: Sat, 05 Jul 2008 19:36:22 GMT
Server: Rocket WSGI Server
Content-Type: text/html
Via: 1.1 127.0.0.1:8000
Connection: close
Transfer-Encoding: chunked

my message

The first argument of HTTP is the HTTP status code. The second argument is the string that will be returned as the body of the response. Additional optional named arguments are used to build the response HTTP header. For example:

1.
raise HTTP(400, 'my message', test='hello')

generates:

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
HTTP/1.1 400 BAD REQUEST
Date: Sat, 05 Jul 2008 19:36:22 GMT
Server: Rocket WSGI Server
Content-Type: text/html
Via: 1.1 127.0.0.1:8000
Connection: close
Transfer-Encoding: chunked
test: hello

my message

If you do not want to commit the open database transaction, rollback before raising the exception.

Any exception other than HTTP causes web2py to roll back any open database transaction, log the error traceback, issue a ticket to the visitor, and return a standard error page.

This means that only HTTP can be used for cross-page control flow. Other exceptions must be caught by the application, otherwise they are ticketed by web2py.

The command:

1.
redirect('http://www.web2py.com')

is simply a shortcut for:

1.
2.
3.
raise HTTP(303,
'You are being redirected <a href="%s">here</a>' % location,
Location='http://www.web2py.com')

The named arguments of the HTTP initializer method are translated into HTTP header directives, in this case, the redirection target location. redirect takes an optional second argument, which is the HTTP status code for the redirection (303 by default). Change this number to 307 for a temporary redirect or to 301 for a permanent redirect.

The most common way to use redirect is to redirect to other pages in the same app and (optionally) pass parameters:

1.
redirect(URL('index', args=(1,2,3), vars=dict(a='b')))

T and Internationalization

T
internationalization

The object T is the language translator. It constitutes a single global instance of the web2py class gluon.language.translator. All string constants (and only string constants) should be marked by T, for example:

1.
a = T("hello world")

Strings that are marked with T are identified by web2py as needing language translation and they will be translated when the code (in the model, controller, or view) is executed. If the string to be translated is not a constant but a variable, it will be added to the translation file at runtime (except on GAE) to be translated later.

The T object can also contain interpolated variables and support multiple equivalent syntax:

1.
2.
3.
4.
a = T("hello %s", ('Tim',))
a = T("hello %(name)s", dict(name='Tim'))
a = T("hello %s") % ('Tim',)
a = T("hello %(name)s") % dict(name='Tim')

The latter syntax is recommended because it makes translation easier. The first string is translated according to the requested language file and the name variable is replaced independently of the language.

Concatenating translation translated strings and normal strings is possible:

1.
T("blah ") + name + T(" blah")   # invalid!

but the opposite no:

1.
name + T(" blah")   # invalid!

The following code is also allowed and often preferable:

1.
T("blah %(name)s blah", dict(name='Tim'))

or the alternative syntax

1.
T("blah %(name)s blah") % dict(name='Tim')

In both cases the translation occurs before the variable name is substituted in the "%(name)s" slot. The following alternative should NOT BE USED:

1.
T("blah %(name)s blah" % dict(name='Tim'))

because translation would occur after substitution.

The requested language is determined by the "Accept-Language" field in the HTTP header, but this selection can be overwritten programmatically by requesting a specific file, for example:

1.
T.force('it-it')

which reads the "languages/it-it.py" language file. Language files can be created and edited via the administrative interface.

You can also force a per-string language:

1.
T("Hello World", language="it-it")

You can turn off translations completely via

1.
T.force(None)

Normally, string translation is evaluated lazily when the view is rendered; hence, the translator force method should not be called inside a view.

It is possible to disable lazy evaluation via

1.
T.lazy = False

In this way, strings are translated immediately by the T operator based on the currently accepted or forced language.

It is also possible to disable lazy evaluation for individual strings:

1.
T("Hello World", lazy=False)

A common issue is the following. The original application is in English. Suppose that there is a translation file (for example Italian, "it-it.py") and the HTTP client declares that it accepts both English (en) and Italian (it-it) in that order. The following unwanted situation occurs: web2py does not know the default is written in English (en). Therefore, it prefers translating everything into Italian (it-it) because it only found the Italian translation file. If it had not found the "it-it.py" file, it would have used the default language strings (English).

There are two solutions for this problem: create a translation language for English, which would be redundant and unnecessary, or better, tell web2py which languages should use the default language strings (the strings coded into the application). This can be done with:

1.
T.set_current_languages('en', 'en-en')

It stores in T.current_languages a list of languages that do not require translation and forces a reload of the language files.

Notice that "it" and "it-it" are different languages from the point of view of web2py. To support both of them, one would need two translation files, always lower case. The same is true for all other languages.

The currently accepted language is stored in

1.
T.accepted_language

Mind that T(...) does not just translate strings but can also translated variables:

1.
2.
>>> a="test"
>>> print T(a)

In this case the word "test" in translated but, if not found and if the filesystem is writable, it will add it to the list of words to be translated in the language file.

Cookies

cookies

web2py uses the Python cookies modules for handling cookies.

Cookies from the browser are in request.cookies and cookies sent by the server are in response.cookies.

You can set a cookie as follows:

1.
2.
3.
response.cookies['mycookie'] = 'somevalue'
response.cookies['mycookie']['expires'] = 24 * 3600
response.cookies['mycookie']['path'] = '/'

The second line tells the browser to keep the cookie for 24 hours. The third line tells the browser to send the cookie back to any application (URL path) at the current domain.

The cookie can be made secure with:

1.
response.cookies['mycookie']['secure'] = True

This tells the browser only to send the cookie back over HTTPS and not over HTTP.

The cookie can be retrieved with:

1.
2.
if request.cookies.has_key('mycookie'):
value = request.cookies['mycookie'].value

Unless sessions are disabled, web2py, under the hood, sets the following cookie and uses it to handle sessions:

1.
2.
response.cookies[response.session_id_name] = response.session_id
response.cookies[response.session_id_name]['path'] = "/"

Note, if a single application includes multiple subdomains, and you want to share the session across those subdomains (e.g., sub1.yourdomain.com, sub2.yourdomain.com, etc.), you must explicitly set the domain of the session cookie as follows:

1.
2.
if not request.env.remote_addr in ['127.0.0.1', 'localhost']:
response.cookies[response.session_id_name]['domain'] = ".yourdomain.com"

The above can be useful if, for example, you want to allow the user to remain logged in across subdomains.

Application init

init

When you deploy web2py, you will want to set a default application, i.e., the application that starts when there is an empty path in the URL, as in:

1.
http://127.0.0.1:8000

By default, when confronted with an empty path, web2py looks for an application called init. If there is no init application it looks for an application called welcome.

default_application
The name of the default application can be changed from init to another name by setting default_application in routes.py:
1.
default_application = "myapp"

Note: default_application first appeared in web2py version 1.83.

Here are four ways to set the default application:

URL rewrite

url rewrite
routes_in
routes_out

web2py has the ability to rewrite the URL path of incoming requests prior to calling the controller action (URL mapping), and conversely, web2py can rewrite the URL path generated by the URL function (reverse URL mapping). One reason to do this is for handling legacy URLs, another is to simplify paths and make them shorter.

web2py includes two distinct URL rewrite systems: an easy-to-use parameter-based system for most use cases, and a flexible pattern-based system for more complex cases. To specify the URL rewrite rules, create a new file in the "web2py" folder called routes.py (the contents of routes.py will depend on which of the two rewrite systems you choose, as described in the next two sections). The two systems cannot be mixed.

Notice that if you edit routes.py, you must reload it. This can be done in two ways: by restarting the web server or by clicking on the routes reload button in admin. If there is a bug in routes, they will not reload.

Parameter-based system

The parameter-based (parametric) router provides easy access to several "canned" URL-rewrite methods. Its capabilities include:

* Omitting default application, controller and function names from externally-visible URLs (those created by the URL() function)

* Mapping domains (and/or ports) to applications or controllers

* Embedding a language selector in the URL

* Removing a fixed prefix from incoming URLs and adding it back to outgoing URLs

* Mapping root files such as /robots.txt to an applications static directory

The parametric router also provides somewhat more flexible validation of incoming URLs.

Suppose you've written an application called myapp and wish to make it the default, so that the application name is no longer part of the URL as seen by the user. Your default controller is still default, and you want to remove its name from user-visible URLs as well. Here's what you put in routes.py:

1.
2.
3.
routers = dict(
BASE = dict(default_application='myapp'),
)

That's it. The parametric router is smart enough to know how to do the right thing with URLs such as:

1.
http://domain.com/myapp/default/myapp
or
1.
http://domain.com/myapp/myapp/index
where normal shortening would be ambiguous. If you have two applications, myapp and myapp2, you'll get the same effect, and additionally myapp2's default controller will be stripped from the URL whenever it's safe (which is mostly all the time).

Here is another case: suppose you want to support URL-based languages, where your URLs look like this:

1.
http://myapp/en/some/path
or (rewritten)
1.
http://en/some/path

Here's how:

1.
2.
3.
4.
routers = dict(
BASE = dict(default_application='myapp'),
myapp = dict(languages=['en', 'it', 'jp'], default_language='en'),
)

Now an incoming URL like this:

1.
http:/domain.com/it/some/path
will be routed to /myapp/some/path, and request.uri_language will be set to 'it', so you can force the translation. You can also have language-specific static files.

1.
http://domain.com/it/static/filename
will be mapped to:
1.
applications/myapp/static/it/filename
if that file exists. If it doesn't, then URLs like:
1.
http://domain.com/it/static/base.css
will still map to:
1.
applications/myapp/static/base.css
(because there is no static/it/base.css).

So you can now have language-specific static files, including images, if you need to. Domain mapping is supported as well:

1.
2.
3.
4.
5.
6.
7.
8.
routers = dict(
BASE = dict(
domains = {
'domain1.com' : 'app1',
'domain2.com' : 'app2',
}
),
)
does what you'd expect.

1.
2.
3.
4.
5.
6.
7.
8.
routers = dict(
BASE = dict(
domains = {
'domain.com:80' : 'app/insecure',
'domain.com:443' : 'app/secure',
}
),
)
maps http://domain.com accesses to the controller named insecure, while HTTPS accesses go to the secure controller. Alternatively, you can map different ports to different apps, in the obvious way.

For further information, please consult the file router.example.py provided in the base folder of the standard web2py distribution.

Note: The parameter-based system first appeared in web2py version 1.92.1.

Pattern-based system

Although the parameter-based system just described should be sufficient for most use cases, the alternative pattern-based system provides some additional flexibility for more complex cases. To use the pattern-based system, instead of defining routers as dictionaries of routing parameters, you define two lists (or tuples) of 2-tuples, routes_in and routes_out. Each tuple contains two elements: the pattern to be replaced and the string that replaces it. For example:

1.
2.
3.
4.
5.
6.
routes_in = (
(
'/testme', '/examples/default/index'),
)
routes_out = (
(
'/examples/default/index', '/testme'),
)

With these routes, the URL:

1.
http://127.0.0.1:8000/testme

is mapped into:

1.
http://127.0.0.1:8000/examples/default/index

To the visitor, all links to the page URL looks like /testme.

The patterns have the same syntax as Python regular expressions. For example:

1.
  ('.*\.php', '/init/default/index'),

maps all URLs ending in ".php" to the index page.

Sometimes you want to get rid of the application prefix from the URLs because you plan to expose only one application. This can be achieved with:

1.
2.
3.
4.
5.
6.
routes_in = (
(
'/(?P<any>.*)', '/init/\g<any>'),
)
routes_out = (
(
'/init/(?P<any>.*)', '/\g<any>'),
)

There is also an alternative syntax that can be mixed with the regular expression notation above. It consists of using $name instead of (?P<name>\w+) or \g<name>. For example:

1.
2.
3.
4.
5.
6.
7.
routes_in = (
(
'/$c/$f', '/init/$c/$f'),
)

routes_out = (
(
'/init/$c/$f', '/$c/$f'),
)

would also eliminate the "/example" application prefix in all URLs.

Using the $name notation, you can automatically map routes_in to routes_out, provided you don't use any regular expressions. For example:

1.
2.
3.
4.
5.
routes_in = (
(
'/$c/$f', '/init/$c/$f'),
)

routes_out = [(x, y) for (y, x) in routes_in]

If there are multiple routes, the first to match the URL is executed. If no pattern matches, the path is left unchanged.

You can use $anything to match anything (.*) until the end of the line.

Here is a minimal "routes.py" for handling favicon and robots requests:

favicon
robots
1.
2.
3.
4.
5.
routes_in = (
(
'/favicon.ico', '/examples/static/favicon.ico'),
(
'/robots.txt', '/examples/static/robots.txt'),
)
routes_out = ()

Here is a more complex example that exposes a single app "myapp" without unnecessary prefixes but also exposes admin, appadmin and static:

1.
2.
3.
4.
5.
6.
7.
8.
routes_in = (
(
'/admin/$anything', '/admin/$anything'),
(
'/static/$anything', '/myapp/static/$anything'),
(
'/appadmin/$anything', '/myapp/appadmin/$anything'),
(
'/favicon.ico', '/myapp/static/favicon.ico'),
(
'/robots.txt', '/myapp/static/robots.txt'),
)
routes_out = [(x, y) for (y, x) in routes_in[:-2]]

The general syntax for routes is more complex than the simple examples we have seen so far. Here is a more general and representative example:

1.
2.
3.
4.
routes_in = (
(
'140\.191\.\d+\.\d+:https://www.web2py.com:POST /(?P<any>.*)\.php',
'/test/default/index?vars=\g<any>'),
)

It maps https POST requests to host www.web2py.com from a remote IP matching the regular expression

1.
'140\.191\.\d+\.\d+'

requesting a page matching the regular expression

1.
'/(?P<any>.*)\.php'

into

1.
'/test/default/index?vars=\g<any>'

where \g<any> is replaced by the matching regular expression.

The general syntax is

1.
'[remote address]:[protocol]://[host]:[method] [path]'

The entire expression is matched as a regular expression, so "." should always be escaped and any matching subexpression can be captured using (?P<...>...) according to Python regex syntax.

This allows to reroute requests based on the client IP address or domain, based on the type of the request, on the method, and the path. It also allows web2py to map different virtual hosts into different applications. Any matched subexpression can be used to build the target URL and, eventually, passed as a GET variable.

All major web servers, such as Apache and lighttpd, also have the ability to rewrite URLs. In a production environment that may be an option instead of routes.py. Whatever you decide to do we strongly suggest that you do not hardcode internal URLs in your app and use the URL function to generate them. This will make your application more portable in case routes should change.

Application-Specific URL rewrite

routes_app
When using the pattern-based system, an application can set its own routes in an application-specific routes.py file located in the applications base folder. This is enabled by configuring routes_app in the base routes.py to determine from an incoming URL the name of the application to be selected. When this happens, the application-specific routes.py is used in place of the base routes.py.

The format of routes_app is identical to routes_in, except that the replacement pattern is simply the application name. If applying routes_app to the incoming URL does not result in an application name, or the resulting application-specific routes.py is not found, the base routes.py is used as usual.

Note: routes_app first appeared in web2py version 1.83.

Default application, controller, and function

default_application
default_controller
default_function

When using the pattern-based system, the name of the default application, controller, and function can be changed from init, default, and index respectively to another name by setting the appropriate value in routes.py:

1.
2.
3.
default_application = "myapp"
default_controller = "admin"
default_function = "start"

Note: These items first appeared in web2py version 1.83.

Routes on error

routes_onerror

You can also use routes.py to re-route requests to special actions in case there is an error on the server. You can specify this mapping globally, for each app, for each error code, or for each app and error code. Here is an example:

1.
2.
3.
4.
5.
6.
routes_onerror = [
(
'init/400', '/init/default/login'),
(
'init/*', '/init/static/fail.html'),
(
'*/404', '/init/static/cantfind.html'),
(
'*/*', '/init/error/index')
]

For each tuple, the first string is matched against "[app name]/[error code]". If a match is found, the failed request is re-routed to the URL in the second string of the matching tuple. If the error handling URL is a not a static file, the following GET variables will be passed to the error action:

These variables will be accessible to the error handling action via request.vars and can be used in generating the error response. In particular, it is a good idea for the error action to return the original HTTP error code instead of the default 200 (OK) status code. This can be done by setting response.status = request.vars.code. It is also possible to have the error action send (or queue) an email to an administrator, including a link to the ticket in admin.

Unmatched errors display a default error page. This default error page can also be customized here (see router.example.py and routes.example.py in the root web2py folder):

1.
2.
3.
4.
error_message = '<html><body><h1>%s</h1></body></html>'
error_message_ticket = '''<html><body><h1>Internal error</h1>
Ticket issued: <a href="/admin/default/ticket/%(ticket)s"
target="_blank">%(ticket)s</a></body></html>'''

The first variable contains the error message when an invalid application or function is requested. The second variable contains the error message when a ticket is issued.

routes_onerror work with both routing mechanisms

Running tasks in the background

In web2py, every http request is served in its own thread. Threads are recycled for efficiency and managed by the web server. For security, the web server sets a time-out on each request. This means that actions should not run tasks that take too long, should not create new threads, and should not fork processes (it is possible but not recommended).

The proper way to run time-consuming tasks is doing it in the background. There is not a single was of doing it, but here we describe three mechanisms that are built into web2py: cron, homemade task queues, and scheduler.

By cron we refer to a web2py functionality not to the Unix Cron mechanism. The web2py cron works on windows too.

web2py cron is the way to go if you need tasks in the background at scheduled times and these tasks take a relatively short time compared to the time interval between two calls. Each task runs in its own process, and multiple tasks can run concurrently, but you have no control over how many tasks run. If accidentally one task overlaps with itself, it can cause a database lock and a spike in memory usage.

web2py scheduler takes a different approach. The number of running processes is fixed, and they can run on different machines. Each process is called a worker. Each worker picks a task when available and executes it as soon as possible after the time when it is scheduled to run, but not necessarily at that exact time. There cannot be more processes running than the number of scheduled tasks and therefore no memory spikes. Scheduler tasks can be defined in models and are stored in the database. The web2py scheduler does not implement a distributed queue since it assumes that the time to distribute tasks is negligible compared with the time to run the tasks. Workers pick up the task from the database.

Homemade tasks queues can be a simpler alternative to the web2py scheduler in some cases.

Cron

cron

The web2py cron provides the ability for applications to execute tasks at preset times, in a platform-independent manner.

For each application, cron functionality is defined by a crontab file:

app/cron/crontab

It follows the syntax defined in ref. [cron] (with some extensions that are specific to web2py).

This means that every application can have a separate cron configuration and that cron config can be changed from within web2py without affecting the host OS itself.

Here is an example:

1.
2.
3.
4.
5.
0-59/1  *  *  *  *  root python /path/to/python/script.py
30 3 * * * root *applications/admin/cron/db_vacuum.py
*/30 * * * * root **applications/admin/cron/something.py
@reboot root *mycontroller/myfunction
@hourly root *applications/admin/cron/expire_sessions.py

The last two lines in this example use extensions to regular cron syntax to provide additional web2py functionality.

The file "applications/admin/cron/expire_sessions.py" actually exists and ships with the admin app. It checks for expired sessions and deletes them. "applications/admin/cron/crontab" runs this task hourly.
If the task/script is prefixed with an asterisk (*) and ends with .py, it will be executed in the web2py environment. This means you will have all the controllers and models at your disposal. If you use two asterisks (**), the MODELs will not be executed. This is the recommended way of calling, as it has less overhead and avoids potential locking problems.

Notice that scripts/functions executed in the web2py environment require a manual db.commit() at the end of the function or the transaction will be reverted.

web2py does not generate tickets or meaningful tracebacks in shell mode, which is how cron is run, so make sure that your web2py code runs without errors before you set it up as a cron task as you will likely not be able to see those errors when run from cron. Moreover, be careful how you use models: while the execution happens in a separate process, database locks have to be taken into account in order to avoid pages waiting for cron tasks that may be blocking the database. Use the ** syntax if you don't need to use the database in your cron task.

You can also call a controller function, in which case there is no need to specify a path. The controller and function will be that of the invoking application. Take special care about the caveats listed above. Example:

1.
*/30  *  *  *  *  root *mycontroller/myfunction

If you specify @reboot in the first field in the crontab file, the given task will be executed only once, at web2py startup. You can use this feature if you want to pre-cache, check, or initialize data for an application on web2py startup. Note that cron tasks are executed in parallel with the application --- if the application is not ready to serve requests until the cron task is finished, you should implement checks to reflect this. Example:

1.
@reboot  *  *  *  *  root *mycontroller/myfunction

Depending on how you are invoking web2py, there are four modes of operation for web2py cron.

The default is hard cron if you are using the built-in web server; in all other cases, the default is soft cron. Soft cron is the default method if you are using CGI, FASTCGI or WSGI (but note that soft cron is not enabled by default in the standard wsgihandler.py file provided with web2py).

Your tasks will be executed on the first call (page load) to web2py after the time specified in crontab; but only after processing the page, so no delay will be observed by the user. Obviously, there is some uncertainty regarding precisely when the task will be executed, depending on the traffic the site receives. Also, the cron task may get interrupted if the web server has a page load timeout set. If these limitations are not acceptable, see external cron. Soft cron is a reasonable last resort, but if your web server allows other cron methods, they should be preferred over soft cron.

Hard cron is the default if you are using the built-in web server (either directly or via Apache mod_proxy). Hard cron is executed in a parallel thread, so unlike soft cron, there are no limitations with regard to run time or execution time precision.

External cron is not default in any scenario, but requires you to have access to the system cron facilities. It runs in a parallel process, so none of the limitations of soft cron apply. This is the recommended way of using cron under WSGI or FASTCGI.

Example of line to add to the system crontab, (usually /etc/crontab):

1.
0-59/1 * * * * web2py cd /var/www/web2py/ && python web2py.py -J -C -D 1 >> /tmp/cron.output 2>&1

If you are running external cron, make sure you add the -N command line parameter to your web2py startup script or config so there is no collision of multiple types of cron. Also, with external cron, make sure to add either -J (or --cronjob, which is the same) as indicated above so that web2py knows that task is executed by cron. Web2py sets this internally with soft and hard cron.

In cases where you do not need any cron functionality within a particular process, you can use the -N command line parameter to disable it. Note that this might disable some maintenance tasks (like the automatic cleaning of session folders). The most common use of this function is when you:

Homemade task queues

While cron is useful to run tasks at regular time intervals, it is not always the best solution to run a background task. For this purpose web2py provides the ability to run any python script as if it were inside a controller:

1.
python web2py.py -S app -M -N -R applications/app/private/myscript.py -A a b c

where -S app tells web2py to run "myscript.py" as "app", -M tells web2py to execute models, -N tells web2py not to run cron, and -A a b c passes optional command line arguments sys.args=['a','b','c'] to "myscript.py".

This type of background process should not be executed via cron (except perhaps for cron @reboot) because you need to be sure that no more than one instance is running at the same time. With cron it is possible that a process starts at cron iteration 1 and is not completed by cron iteration 2, so cron starts it again, and again, and again - thus jamming the mail server.

In chapter 8, we will provide an example of how to use the above method to send emails.

Scheduler (experimental)

The web2py scheduler works very much like the task queue described in the previous sub-section with some differences:

The scheduler does not use cron, although one can use cron @reboot to start the worker nodes.

In the scheduler, a task is simply a function defined in a model (or in a module and imported by a model). For example:

1.
2.
def task_add(a,b):
return a+b

Tasks will always be called in the same environment seen by controllers and therefore they see all the global variables defined in models, including database connections (db). Tasks differ from a controller action because they are not associated with an HTTP request and therefore there is no request.env.

Once the tasks are defined, you need to enable the scheduler by adding the following code to your model:

myscheduler = Scheduler(db, dict(task_add=task_add))

The first argument of the Scheduler class must be the database to be used by the scheduler to communicate with the workers. This can be the db of the app or another dedicated db, perhaps one shared by multiple apps. The Scheduler creates the tables it needs. The second argument is a Python dictionary of key:value pairs where key is the name you want to use to expose a task, and value is the actual name of the function that defines the task.

Once the tasks are defined and the Scheduler is instantiated, all that is needed to do is to start the workers:

python web2py.py -K myapp

The -K option starts a worker. The argument of the -K option is a list of app names, separated by commas. These are the apps that will be served by this worker. One can start many workers.

Now we have the infrastructure in place: defined the tasks, told the scheduler about them, started the worker(s). What remains is to actually schedule the tasks:

Tasks can be scheduled programmatically or via appadmin. In fact, a task is scheduled simply by adding an entry in the table "scheduler_task", which you can access via appadmin:

http://127.0.0.1:8000/scheduler/appadmin/insert/db/scheduler_task

The meaning of the fields in this table is obvious. The "args" and "vars"" fields are the values to be passed to the task in JSON format. In the case of the "task_add" above, an example of "args" and "vars" could be:

args = [3, 4]
vars = {}

or

args = []
vars = {'a':3, 'b':4}

A task can be in one of the following states:

QUEUED, RUNNING, COMPLETED, FAILED, TIMEOUT

Once a task exists (there is a record in the "scheduler_task" table), is QUEUED, and is ready (meets all the conditions specified in the record), it can be picked up by a worker. As soon as a worker is available, it picks the first ready task scheduled to run. The worker creates an entry in a another table, "scheduler_run" (also created by the scheduler).

The table "scheduler_run" stores the status of all running tasks. Each record references a task that has been picked up by a worker. One task can have multiple runs. For example, a task scheduled to repeat 10 times an hour will probably have 10 runs (unless one fails or they take longer than 1 hour).

Possible run statuses are:

RUNNING, COMPLETED, FAILED, TIMEOUT

When a QUEUED task is picked up, it becomes a RUNNING task and its run status is also RUNNING. If the run is completed, no exceptions are thrown, and there is no task timeout, the run is marked as COMPLETED and the task is marked as QUEUED or COMPLETED depending on whether it is supposed to run again at a later time. The output of the task is serialized in JSON and stored in the run record.

When a RUNNING task throws an exception, the run is mark as FAILED and the task is marked as FAILED. The traceback is stored in the run record.

Similarly, when a run exceeds the timeout, it is stopped and marked as TIMEOUT, and the task is marked as TIMEOUT.

In any case, the stdout is captured and also logged into the run record.

Using appadmin, one can check all RUNNING tasks, the output of COMPLETED tasks, the error of FAILED tasks, etc.

The scheduler also creates one more table called "scheduler_worker", which stores the workers' heartbeat and their status. Possible worker statuses are:

ACTIVE, INACTIVE, DISABLED

You can disable a worker by changing its status using appadmin.

Everything that one can do via appadmin one can do programmatically by inserting and updating records in these tables.

Anyway, one should not update records relative to RUNNING tasks as this may create an un-expected behavior. The best practice is to queue tasks using "insert". For example:

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
db.scheduler_task.insert(
status='QUEUED',
application_name='myapp',
task_name='my first task',
function_name='task_add',
args='[]',
vars="{'a':3,'b':4}",
enabled=True,
start_time = request.now,
stop_time = request.now+datetime.timedelta(days=1),
repeats = 10, # run 10 times
period = 3600, # every 1h
timeout = 60, # should take less than 60 seconds
)

Notice that fields "times_run", "last_run_time" and "assigned_worker_name" are not provided at schedule time but are filled automatically by the workers.

You can also retrieve the output of completed tasks:

1.
completed_runs = db(db.scheduler_run.status='COMPLETED').select()
The scheduler is experimental because it needs more extensive testing and because the table structure may change as more features are added.

Third party modules

import

web2py is written in Python, so it can import and use any Python module, including third party modules. It just needs to be able to find them. As with any Python application, modules can be installed in the official Python "site-packages" directory, and they can then be imported from anywhere inside your code.

Modules in the "site-packages" directory are, as the name suggests, site-level packages. Applications requiring site-packages are not portable unless these modules are installed separately. The advantage of having modules in "site-packages" is that multiple applications can share them. Let's consider, for example, the plotting package called "matplotlib". You can install it from the shell using the PEAK easy_install command:

1.
easy_install py-matplotlib

and then you can import it into any model/controller/view with:

1.
import matplotlib

The web2py source distribution, and the Windows binary distribution has a site-packages in the top-level folder. The Mac binary distribution has a site-packages folder in the folder:

1.
web2py.app/Contents/Resources/site-packages

The problem with using site-packages is that it becomes difficult to use different versions of a single module at the same time, for example there could be two applications but each one uses a different version of the same file. In this example, sys.path cannot be altered because it would affect both applications.

For this kind of situation, web2py provides another way to import modules in such a way that the global sys.path is not altered: by placing them in the "modules" folder of an application. One side benefit is that the module will be automatically copied and distributed with the application.

Once a module "mymodule.py" is placed into an app "modules/" folder, it can be imported from anywhere inside a web2py application (without need to alter sys.path with):
import mymodule

Execution environment

exec_environment
While everything discussed here works fine, we recommend instead building your application using components, as described in chapter 12.
web2py model and controller files are not Python modules in that they cannot be imported using the Python import statement. The reason for this is that models and controllers are designed to be executed in a prepared environment that has been pre-populated with web2py global objects (request, response, session, cache and T) and helper functions. This is necessary because Python is a statically (lexically) scoped language, whereas the web2py environment is created dynamically.

web2py provides the exec_environment function to allow you to access models and controllers directly. exec_environment creates a web2py execution environment, loads the file into it and then returns a Storage object containing the environment. The Storage object also serves as a namespace mechanism. Any Python file designed to be executed in the execution environment can be loaded using exec_environment. Uses for exec_environment include:

This example reads rows from the user table in the cas application:

1.
2.
3.
from gluon.shell import exec_environment
cas = exec_environment('applications/cas/models/db.py')
rows = cas.db().select(cas.db.user.ALL)

Another example: suppose you have a controller "other.py" that contains:

1.
2.
def some_action():
return dict(remote_addr=request.env.remote_addr)

Here is how you can call this action from another controller (or from the web2py shell):

1.
2.
3.
from gluon.shell import exec_environment
other = exec_environment('applications/app/controllers/other.py', request=request)
result = other.some_action()

In line 2, request=request is optional. It has the effect of passing the current request to the environment of "other". Without this argument, the environment would contain a new and empty (apart from request.folder) request object. It is also possible to pass a response and a session object to exec_environment. Be careful when passing request, response and session objects --- modification by the called action or coding dependencies in the called action could lead to unexpected side effects.

The function call in line 3 does not execute the view; it simply returns the dictionary unless response.render is called explicitly by "some_action".

One final caution: don't use exec_environment inappropriately. If you want the results of actions in another application, you probably should implement an XML-RPC API (implementing an XML-RPC API with web2py is almost trivial). Don't use exec_environment as a redirection mechanism; use the redirect helper.

Cooperation

cooperation

There are many ways applications can cooperate:

1.
from applications.appname.modules import mymodule

One app can load the session of another app using the command:

1.
session.connect(request, response, masterapp='appname', db=db)

Here "appname" is the name of the master application, the one that sets the initial session_id in the cookie. db is a database connection to the database that contains the session table (web2py_session). All apps that share sessions must use the same database for session storage.

One application can load a module from another app using

1.
import applications.otherapp.modules.othermodule

Logging

Python provides logging APIs. Web2py provides a mechanism to configure it so that apps can use it.

In your application, you can create a logger, for example in a model:

1.
2.
3.
import logging
logger = logging.getLogger("web2py.app.myapp")
logger.setLevel(logging.DEBUG)

and you can use it to log messages of various importance

1.
2.
3.
4.
logger.debug("Just checking that %s" % details)
logger.info("You ought to know that %s" % details)
logger.warn("Mind that %s" % details)
logger.error("Oops, something bad happened %s" % details)

logging is a standard python module described here:

http://docs.python.org/library/logging.html
The string "web2py.app.myapp" defines an app-level logger.

For this to work properly, you need a configuration file for the logger. One is provided by web2py in the root web2py folder "logging.example.conf". You need to rename the file "logging.conf" and customize it as necessary.

This file is self documenting, so you should open it and read it.

To create a configurable logger for application "myapp", you must add myapp to the [loggers] keys list:

1.
2.
[loggers]
keys=root,rocket,markdown,web2py,rewrite,app,welcome,myapp

and you must add a [logger_myapp] section, using [logger_welcome] as a starting point.

1.
2.
3.
4.
5.
[logger_myapp]
level=WARNING
qualname=web2py.app.myapp
handlers=consoleHandler
propagate=0

The "handlers" directive specifies the type of logging and here it is logging "myapp" to the console.

WSGI

WSGI

web2py and WSGI have a love-hate relationship. Our perspective is that WSGI was developed as a protocol to connect web servers to web applications in a portable way, and we use it for that purpose. web2py at its core is a WSGI application: gluon.main.wsgibase. Some developers have pushed WSGI to its limits as a protocol for middleware communications and develop web applications as an onion with many layers (each layer being a WSGI middleware developed independently of the entire framework). web2py does not adopt this structure internally. This is because we feel the core functionality of a frameworks (handling cookies, session, errors, transactions, dispatching) can be better optimized for speed and security if they are handled by a single comprehensive layer.

Yet web2py allows you to use third party WSGI applications and middleware in three ways (and their combinations):

The only limitation is that you cannot use third party middleware to replace core web2py functions.

External middleware

Consider the file "wsgibase.py":

1.
2.
3.
4.
5.
6.
7.
8.
9.
#...
LOGGING = False
#...
if LOGGING:
application = gluon.main.appfactory(wsgiapp=gluon.main.wsgibase,
logfilename='httpserver.log',
profilerfilename=None)
else:
application = gluon.main.wsgibase

When LOGGING is set to True, gluon.main.wsgibase is wrapped by the middleware function gluon.main.appfactory. It provides logging to the "httpserver.log" file. In a similar fashion you can add any third party middleware. We refer to the official WSGI documentation for more details.

Internal middleware

Given any action in your controllers (for example index) and any third party middleware application (for example MyMiddleware, which converts output to upper case), you can use a web2py decorator to apply the middleware to that action. Here is an example:

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
class MyMiddleware:
"""converts output to upper case"""
def __init__(self,app):
self.app = app
def __call__(self, environ, start_response):
items = self.app(environ, start_response)
return [item.upper() for item in items]

@request.wsgi.middleware(MyMiddleware)
def index():
return 'hello world'

We cannot promise that all third party middleware will work with this mechanism.

Calling WSGI applications

It is easy to call WSGI app from a web2py action. Here is an example:

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
def test_wsgi_app(environ, start_response):
"""this is a test WSGI app"""
status = '200 OK'
response_headers = [('Content-type','text/plain'),
(
'Content-Length','13')]
start_response(status, response_headers)
return ['hello world!\n']

def index():
"""a test action that calls the previous app and escapes output"""
items = test_wsgi_app(request.wsgi.environ,
request.wsgi.start_response)
for item in items:
response.write(item,escape=False)
return response.body.getvalue()

In this case, the index action calls test_wsgi_app and escapes the returned value before returning it. Notice that index is not itself a WSGI app and it must use the normal web2py API (such as response.write to write to the socket).

The views

views
template language
HTML

web2py uses Python for its models, controllers, and views, although it uses a slightly modified Python syntax in the views to allow more readable code without imposing any restrictions on proper Python usage.

The purpose of a view is to embed code (Python) in an HTML document. In general, this poses some problems:

web2py uses {{ ... }} to escape Python code embedded in HTML. The advantage of using curly brackets instead of angle brackets is that it's transparent to all common HTML editors. This allows the developer to use those editors to create web2py views.

Since the developer is embedding Python code into HTML, the document should be indented according to HTML rules, and not Python rules. Therefore, we allow unindented Python inside the {{ ... }} tags. Since Python normally uses indentation to delimit blocks of code, we need a different way to delimit them; this is why the web2py template language makes use of the Python keyword pass.

A code block starts with a line ending with a colon and ends with a line beginning with pass. The keyword pass is not necessary when the end of the block is obvious from the context.
Here is an example:
1.
2.
3.
4.
5.
6.
7.
{{
if i == 0:
response.write('i is 0')
else:
response.write('i is not 0')
pass
}}

Note that pass is a Python keyword, not a web2py keyword. Some Python editors, such as Emacs, use the keyword pass to signify the division of blocks and use it to re-indent code automatically.

The web2py template language does exactly the same. When it finds something like:

1.
2.
3.
<html><body>
{{
for x in range(10):}}{{=x}}hello<br />{{pass}}
</body></html>

it translates it into a program:

1.
2.
3.
4.
5.
response.write("""<html><body>""", escape=False)
for x in range(10):
response.write(x)
response.write("""hello<br />""", escape=False)
response.write("""</body></html>""", escape=False)
response.write writes to the response.body.

When there is an error in a web2py view, the error report shows the generated view code, not the actual view as written by the developer. This helps the developer debug the code by highlighting the actual code that is executed (which is something that can be debugged with an HTML editor or the DOM inspector of the browser).

Also note that:

1.
{{=x}}

generates

response.write
escape
1.
response.write(x)

Variables injected into the HTML in this way are escaped by default. The escaping is ignored if x is an XML object, even if escape is set to True.

Here is an example that introduces the H1 helper:

1.
{{=H1(i)}}

which is translated to:

1.
response.write(H1(i))

upon evaluation, the H1 object and its components are recursively serialized, escaped and written to the response body. The tags generated by H1 and inner HTML are not escaped. This mechanism guarantees that all text --- and only text --- displayed on the web page is always escaped, thus preventing XSS vulnerabilities. At the same time, the code is simple and easy to debug.

The method response.write(obj, escape=True) takes two arguments, the object to be written and whether it has to be escaped (set to True by default). If obj has an .xml() method, it is called and the result written to the response body (the escape argument is ignored). Otherwise it uses the object's __str__ method to serialize it and, if the escape argument is True, escapes it. All built-in helper objects (H1 in the example) are objects that know how to serialize themselves via the .xml() method.

This is all done transparently. You never need to (and never should) call the response.write method explicitly.

Basic syntax

The web2py template language supports all Python control structures. Here we provide some examples of each of them. They can be nested according to usual programming practice.

for...in

for

In templates you can loop over any iterable object:

1.
2.
3.
4.
{{items = ['a', 'b', 'c']}}
<ul>
{{for item in items:}}<li>{{=item}}</li>{{pass}}
</ul>

which produces:

1.
2.
3.
4.
5.
<ul>
<
li>a</li>
<
li>b</li>
<
li>c</li>
</
ul>

Here item is any iterable object such as a Python list, Python tuple, or Rows object, or any object that is implemented as an iterator. The elements displayed are first serialized and escaped.

while

while

You can create a loop using the while keyword:

1.
2.
3.
4.
{{k = 3}}
<ul>
{{while k > 0:}}<li>{{=k}}{{k = k - 1}}</li>{{pass}}
</ul>

which produces:

1.
2.
3.
4.
5.
<ul>
<
li>3</li>
<
li>2</li>
<
li>1</li>
</
ul>

if...elif...else

if
elif
else

You can use conditional clauses:

1.
2.
3.
4.
5.
6.
7.
8.
{{
import random
k = random.randint(0, 100)
}}
<h2>
{{=k}}
{{if k % 2:}}is odd{{else:}}is even{{pass}}
</h2>

which produces:

1.
2.
3.
<h2>
45 is odd
</h2>

Since it is obvious that else closes the first if block, there is no need for a pass statement, and using one would be incorrect. However, you must explicitly close the else block with a pass.

Recall that in Python "else if" is written elif as in the following example:

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
{{
import random
k = random.randint(0, 100)
}}
<h2>
{{=k}}
{{if k % 4 == 0:}}is divisible by 4
{{elif k % 2 == 0:}}is even
{{else:}}is odd
{{pass}}
</h2>

It produces:

1.
2.
3.
<h2>
64 is divisible by 4
</h2>

try...except...else...finally

try
except
else
finally

It is also possible to use try...except statements in views with one caveat. Consider the following example:

1.
2.
3.
4.
5.
6.
7.
8.
9.
{{try:}}
Hello {{= 1 / 0}}
{{except:}}
division by zero
{{else:}}
no division by zero
{{finally}}
<br />
{{pass}}

It will produce the following output:

1.
2.
3.
Hello
division by zero
<br />

This example illustrates that all output generated before an exception occurs is rendered (including output that preceded the exception) inside the try block. "Hello" is written because it precedes the exception.

def...return

def
return

The web2py template language allows the developer to define and implement functions that can return any Python object or a text/html string. Here we consider two examples:

1.
2.
3.
4.
{{def itemize1(link): return LI(A(link, _href="http://" + link))}}
<ul>
{{=itemize1('www.google.com')}}
</ul>

produces the following output:

1.
2.
3.
<ul>
<
li><a href="http:/www.google.com">www.google.com</a></li>
</
ul>

The function itemize1 returns a helper object that is inserted at the location where the function is called.

Consider now the following code:

1.
2.
3.
4.
5.
6.
{{def itemize2(link):}}
<li><a href="http://{{=link}}">{{=link}}</a></li>
{{return}}
<ul>
{{itemize2('www.google.com')}}
</ul>

It produces exactly the same output as above. In this case, the function itemize2 represents a piece of HTML that is going to replace the web2py tag where the function is called. Notice that there is no '=' in front of the call to itemize2, since the function does not return the text, but it writes it directly into the response.

There is one caveat: functions defined inside a view must terminate with a return statement, or the automatic indentation will fail.

HTML helpers

helpers

Consider the following code in a view:

1.
{{=DIV('this', 'is', 'a', 'test', _id='123', _class='myclass')}}

it is rendered as:

1.
<div id="123" class="myclass">thisisatest</div>
DIV is a helper class, i.e., something that can be used to build HTML programmatically. It corresponds to the HTML <div> tag.

Positional arguments are interpreted as objects contained between the open and close tags. Named arguments that start with an underscore are interpreted as HTML tag attributes (without the underscore). Some helpers also have named arguments that do not start with underscore; these arguments are tag-specific.

Instead of a set of unnamed arguments, a helper can also take a single list or tuple as its set of components using the * notation and it can take a single dictionary as its set of attributes using the **, for example:

1.
2.
3.
4.
5.
{{
contents = ['this','is','a','test']
attributes = {'_id':'123', '_class':'myclass'}
=
DIV(*contents,**attributes)
}}
(produces the same output as before).

The following set of helpers:

A, B, BEAUTIFY, BODY, BR, CAT, CENTER, CODE, COL, COLGROUP, DIV, EM, EMBED, FIELDSET, FORM, H1, H2, H3, H4, H5, H6, HEAD, HR, HTML, I, IFRAME, IMG, INPUT, LABEL, LEGEND, LI, LINK, MARKMIN, MENU, META, OBJECT, ON, OL, OPTGROUP, OPTION, P, PRE, SCRIPT, SELECT, SPAN, STYLE, TABLE, TAG, TBODY, TD, TEXTAREA, TFOOT, TH, THEAD, TITLE, TR, TT, UL, URL, XHTML, XML, embed64, xmlescape

can be used to build complex expressions that can then be serialized to XML[xml:w][xml:o]. For example:

1.
{{=DIV(B(I("hello ", "<world>"))), _class="myclass")}}

is rendered:

1.
<div class="myclass"><b><i>hello &lt;world&gt;</i></b></div>

Helpers can also be serialized into strings, equivalently, with the __str__ and the xml methods:

>>> print str(DIV("hello world"))
<div>hello world</div>
>>> print DIV("hello world").xml()
<div>hello world</div>

Document Object Model (DOM)
The helpers mechanism in web2py is more than a system to generate HTML without concatenating strings. It provides a server-side representation of the Document Object Model (DOM).

Components of helpers can be referenced via their position, and helpers act as lists with respect to their components:

1.
2.
3.
4.
5.
6.
7.
8.
>>> a = DIV(SPAN('a', 'b'), 'c')
>>>
print a
<div><span>ab</span>c</div>
>>>
del a[1]
>>>
a.append(B('x'))
>>>
a[0][0] = 'y'
>>> print a
<div><span>yb</span><b>x</b></div>

Attributes of helpers can be referenced by name, and helpers act as dictionaries with respect to their attributes:

1.
2.
3.
4.
5.
>>> a = DIV(SPAN('a', 'b'), 'c')
>>>
a['_class'] = 's'
>>> a[0]['_class'] = 't'
>>> print a
<div class="s"><span class="t">ab</span>c</div>

Note, the complete set of components can be accessed via a list called a.components, and the complete set of attributes can be accessed via a dictionary called a.attributes. So, a[i] is equivalent to a.components[i] when i is an integer, and a[s] is equivalent to a.attributes[s] when s is a string.

Notice that helper attributes are passed as keyword arguments to the helper. In some cases, however, attribute names include special characters that are not allowed in Python identifiers (e.g., hyphens) and therefore cannot be used as keyword argument names. For example:

1.
DIV('text', _data-role='collapsible')

will not work because "_data-role" includes a hyphen, which will produce a Python syntax error.

In such cases, you can instead pass the attributes as a dictionary and make use of Python's ** function arguments notation, which map a dictionary of (key:value) pairs into a set of keyword arguments:

1.
2.
>>> print DIV('text', **{'_data-role': 'collapsible'})
<
div data-role="collapsible">text</div>

You can also dynamically create special TAGs:

1.
2.
>>> print TAG['soap:Body']('whatever',**{'_xmlns:m':'http://www.example.org'})
<
soap:Body xmlns:m="http://www.example.org">whatever</soap:Body>

XML

XML
XML is an object used to encapsulate text that should not be escaped. The text may or may not contain valid XML. For example, it could contain JavaScript.

The text in this example is escaped:

1.
2.
>>> print DIV("<b>hello</b>")
&
lt;b&gt;hello&lt;/b&gt;

by using XML you can prevent escaping:

1.
2.
>>> print DIV(XML("<b>hello</b>"))
<
b>hello</b>

Sometimes you want to render HTML stored in a variable, but the HTML may contain unsafe tags such as scripts:

1.
2.
>>> print XML('<script>alert("unsafe!")</script>')
<
script>alert("unsafe!")</script>

Un-escaped executable input such as this (for example, entered in the body of a comment in a blog) is unsafe, because it can be used to generate Cross Site Scripting (XSS) attacks against other visitors to the page.

sanitize
The web2py XML helper can sanitize our text to prevent injections and escape all tags except those that you explicitly allow. Here is an example:
1.
2.
>>> print XML('<script>alert("unsafe!")</script>', sanitize=True)
&
lt;script&gt;alert(&quot;unsafe!&quot;)&lt;/script&gt;

The XML constructors, by default, consider the content of some tags and some of their attributes safe. You can override the defaults using the optional permitted_tags and allowed_attributes arguments. Here are the default values of the optional arguments of the XML helper.

1.
2.
3.
4.
5.
XML(text, sanitize=False,
permitted_tags=['a', 'b', 'blockquote', 'br/', 'i', 'li',
'ol', 'ul', 'p', 'cite', 'code', 'pre', 'img/'],
allowed_attributes={'a':['href', 'title'],
'img':['src', 'alt'], 'blockquote':['type']})

Built-in helpers

A

This helper is used to build links.

A
1.
2.
3.
>>> print A('<click>', XML('<b>me</b>'),
_href='http://www.web2py.com')
<
a href='http://www.web2py.com'>&lt;click&gt;<b>me/b></a>

Instead of _href you can pass the URL using the callback argument. For example in a view:

{{=A('click me', callback=URL('myaction'))}}

and the effect of pressing the link will be an ajax call to "myaction" instead of a redirection. In this case, optionally you can specify two more arguments: target and delete:

{{=A('click me', callback=URL('myaction'), target="t")}}
<div id="t"><div>
and the response of the ajax callback will be stored in the DIV with id equal to "t".

<div id="b">{{=A('click me', callback=URL('myaction'), delete='div#b")}}</div>
and upon response, the closest tag matching "div#b" will be deleted. In this case, the button will be deleted. A typical application is:
{{=A('click me', callback=URL('myaction'), delete='tr")}}

in a table. Pressing the button will perform the callback and delete the table row.

callback and delete can be combined.

Tha A helper takes a special argument called cid. It works as follows:

1.
2.
{{=A('linked page', _href='http://example.com', cid='myid')}}
<div id="myid"></div>

and a click on the link causes the content to be loaded in the div. This is similar but more powerful than the above syntax since it is designed to refresh page components. We discuss applications of cid in more detail in Chapter 12, in the context of components.

These ajax features require jQuery and "static/js/web2py_ajax.js", which are automatically included by placing {{include 'web2py_ajax.html'}} in the layout head. "views/web2py_ajax.html" defines some variables based on request and includes all necessary js and css files.

B
B

This helper makes its contents bold.

1.
2.
>>> print B('<hello>', XML('<i>world</i>'), _class='test', _id=0)
<
b id="0" class="test">&lt;hello&gt;<i>world</i></b>
BODY

BODY
This helper makes the body of a page.
1.
2.
>>> print BODY('<hello>', XML('<b>world</b>'), _bgcolor='red')
<
body bgcolor="red">&lt;hello&gt;<b>world</b></body>
BR
BR

This helper creates a line break.

1.
2.
>>> print BR()
<
br />
CAT (1.98.1 and up)
CAT

This helper concatenates other helpers, same as TAG[''].

1.
2.
>>> print CAT('Here is a ', A('link',_href=URL()), ', and here is some ', B('bold text'), '.')
Here is a <a href="/app/default/index">link</a>, and here is some <b>bold text</b>.
CENTER
CENTER

This helper centers its content.

1.
2.
3.
>>> print CENTER('<hello>', XML('<b>world</b>'),
>>>
_class='test', _id=0)
<
center id="0" class="test">&lt;hello&gt;<b>world</b></center>
CODE
CODE

This helper performs syntax highlighting for Python, C, C++, HTML and web2py code, and is preferable to PRE for code listings. CODE also has the ability to create links to the web2py API documentation.

Here is an example of highlighting sections of Python code.

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
>>> print CODE('print "hello"', language='python').xml()
<
table><tr valign="top"><td style="width:40px; text-align: right;"><pre style="
font-size: 11px;
font-family: Bitstream Vera Sans Mono,monospace;
background-color: transparent;
margin: 0;
padding: 5px;
border: none;
background-color: #E0E0E0;
color: #A0A0A0;
">1.</pre></td><td><pre style="
font-size: 11px;
font-family: Bitstream Vera Sans Mono,monospace;
background-color: transparent;
margin: 0;
padding: 5px;
border: none;
overflow: auto;
"><span style="color:#185369; font-weight: bold">print </span>
<span style="color: #FF9966">"hello"</span></pre></td></tr>
</
table>

Here is a similar example for HTML

1.
2.
3.
>>> print CODE(
>>>
'<html><body>{{=request.env.remote_add}}</body></html>',
>>>
language='html')
1.
2.
3.
<table>...<code>...
<
html><body>{{=request.env.remote_add}}</body></html>
...</code>...</table>

These are the default arguments for the CODE helper:

1.
CODE("print 'hello world'", language='python', link=None, counter=1, styles={})

Supported values for the language argument are "python", "html_plain", "c", "cpp", "web2py", and "html". The "html" language interprets {{ and }} tags as "web2py" code, while "html_plain" doesn't.

If a link value is specified, for example "/examples/global/vars/", web2py API references in the code are linked to documentation at the link URL. For example "request" would be linked to "/examples/global/vars/request". In the above example, the link URL is handled by the "vars" action in the "global.py" controller that is distributed as part of the web2py "examples" application.

The counter argument is used for line numbering. It can be set to any of three different values. It can be None for no line numbers, a numerical value specifying the start number, or a string. If the counter is set to a string, it is interpreted as a prompt, and there are no line numbers.

The styles argument is a bit tricky. If you look at the generated HTML above, it contains a table with two columns, and each column has its own style declared inline using CSS. The styles attributes allows you to override those two CSS styles. For example:

1.
{{=CODE(...,styles={'CODE':'margin: 0;padding: 5px;border: none;'})}}

The styles attribute must be a dictionary, and it allows two possible keys: CODE for the style of the actual code, and LINENUMBERS for the style of the left column, which contains the line numbers. Mind that these styles completely replace the default styles and are not simply added to them.

COL
COL
1.
2.
>>> print COL('a','b')
<
col>ab</col>
COLGROUP
COLGROUP
1.
2.
>>> print COLGROUP('a','b')
<
colgroup>ab</colgroup>
DIV

All helpers apart from XML are derived from DIV and inherit its basic methods.

DIV
1.
2.
>>> print DIV('<hello>', XML('<b>world</b>'), _class='test', _id=0)
<
div id="0" class="test">&lt;hello&gt;<b>world</b></div>
EM

Emphasizes its content.

EM
1.
2.
>>> print EM('<hello>', XML('<b>world</b>'), _class='test', _id=0)
<
em id="0" class="test">&lt;hello&gt;<b>world</b></em>
FIELDSET
FIELDSET

This is used to create an input field together with its label.

1.
2.
>>> print FIELDSET('Height:', INPUT(_name='height'), _class='test')
<
fieldset class="test">Height:<input name="height" /></fieldset>
FORM
FORM

This is one of the most important helpers. In its simple form, it just makes a <form>...</form> tag, but because helpers are objects and have knowledge of what they contain, they can process submitted forms (for example, perform validation of the fields). This will be discussed in detail in Chapter 7.

1.
2.
3.
>>> print FORM(INPUT(_type='submit'), _action='', _method='post')
<
form enctype="multipart/form-data" action="" method="post">
<
input type="submit" /></form>

The "enctype" is "multipart/form-data" by default.

hidden
The constructor of a FORM, and of SQLFORM, can also take a special argument called hidden. When a dictionary is passed as hidden, its items are translated into "hidden" INPUT fields. For example:
1.
2.
3.
>>> print FORM(hidden=dict(a='b'))
<
form enctype="multipart/form-data" action="" method="post">
<
input value="b" type="hidden" name="a" /></form>
H1, H2, H3, H4, H5, H6
H1

These helpers are for paragraph headings and subheadings:

1.
2.
>>> print H1('<hello>', XML('<b>world</b>'), _class='test', _id=0)
<
h1 id="0" class="test">&lt;hello&gt;<b>world</b></h1>
HEAD

For tagging the HEAD of an HTML page.

HEAD
1.
2.
>>> print HEAD(TITLE('<hello>', XML('<b>world</b>')))
<
head><title>&lt;hello&gt;<b>world</b></title></head>
HTML

HTML
XHTML

This helper is a little different. In addition to making the <html> tags, it prepends the tag with a doctype string[xhtml:w,xhtml:o,xhtml:school] .

1.
2.
3.
4.
>>> print HTML(BODY('<hello>', XML('<b>world</b>')))
<!
DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd"
>
<
html><body>&lt;hello&gt;<b>world</b></body></html>

The HTML helper also takes some additional optional arguments that have the following default:

1.
HTML(..., lang='en', doctype='transitional')

where doctype can be 'strict', 'transitional', 'frameset', 'html5', or a full doctype string.

XHTML
XHTML

XHTML is similar to HTML but it creates an XHTML doctype instead.

1.
XHTML(..., lang='en', doctype='transitional', xmlns='http://www.w3.org/1999/xhtml')

where doctype can be 'strict', 'transitional', 'frameset', or a full doctype string.

HR
HR

This helper creates a horizontal line in an HTML page

1.
2.
>>> print HR()
<
hr />
I
I

This helper makes its contents italic.

1.
2.
>>> print I('<hello>', XML('<b>world</b>'), _class='test', _id=0)
<
i id="0" class="test">&lt;hello&gt;<b>world</b></i>
INPUT
INPUT

Creates an <input.../> tag. An input tag may not contain other tags, and is closed by /> instead of >. The input tag has an optional attribute _type that can be set to "text" (the default), "submit", "checkbox", or "radio".

1.
2.
>>> print INPUT(_name='test', _value='a')
<
input value="a" name="test" />

It also takes an optional special argument called "value", distinct from "_value". The latter sets the default value for the input field; the former sets its current value. For an input of type "text", the former overrides the latter:

1.
2.
>>> print INPUT(_name='test', _value='a', value='b')
<
input value="b" name="test" />

For radio buttons, INPUT selectively sets the "checked" attribute:

radio
1.
2.
3.
4.
5.
>>> for v in ['a', 'b', 'c']:
>>>
print INPUT(_type='radio', _name='test', _value=v, value='b'), v
<input value="a" type="radio" name="test" /> a
<input value="b" type="radio" checked="checked" name="test" /> b
<input value="c" type="radio" name="test" /> c

and similarly for checkboxes:

checkbox
1.
2.
3.
4.
>>> print INPUT(_type='checkbox', _name='test', _value='a', value=True)
<
input value="a" type="checkbox" checked="checked" name="test" />
>>>
print INPUT(_type='checkbox', _name='test', _value='a', value=False)
<
input value="a" type="checkbox" name="test" />
IFRAME

This helper includes another web page in the current page. The url of the other page is specified via the "_src" attribute.

IFRAME
1.
2.
>>> print IFRAME(_src='http://www.web2py.com')
<
iframe src="http://www.web2py.com"></iframe>
IMG
IMG

It can be used to embed images into HTML:

1.
2.
>>> IMG(_src='http://example.com/image.png',_alt='test')
<
img src="http://example.com/image.ong" alt="rest" />

Here is a combination of A, IMG, and URL helpers for including a static image with a link:

1.
2.
3.
4.
5.
>>> A(IMG(_src=URL('static','logo.png'), _alt="My Logo"),
_href=URL('default','index'))
<
a href="/myapp/default/index">
<
img src="/myapp/static/logo.png" alt="My Logo" />
</
a>
LABEL

It is used to create a LABEL tag for an INPUT field.

LABEL
1.
2.
>>> print LABEL('<hello>', XML('<b>world</b>'), _class='test', _id=0)
<
label id="0" class="test">&lt;hello&gt;<b>world</b></label>
LEGEND

It is used to create a legend tag for a field in a form.

LEGEND
1.
2.
>>> print LEGEND('Name', _for='myfield')
<
legend for="myfield">Name</legend>
LI

It makes a list item and should be contained in a UL or OL tag.

LI
1.
2.
>>> print LI('<hello>', XML('<b>world</b>'), _class='test', _id=0)
<
li id="0" class="test">&lt;hello&gt;<b>world</b></li>
META

To be used for building META tags in the HTML head. For example:

META
1.
2.
>>> print META(_name='security', _content='high')
<
meta name="security" content="high" />
MARKMIN

Implements the markmin wiki syntax. It converts the input text into output html according to the markmin rules described in the example below:

MARKMIN
1.
2.
3.
>>> print MARKMIN("this is **bold** or ''italic'' and this [[a link http://web2py.com]]")
<
p>this is <b>bold</b> or <i>italic</i> and
this <a href="http://web2py.com">a link</a></p>

The markmin syntax is described in this file that ships with web2py:

1.
http://127.0.0.1:8000/examples/static/markmin.html

and some examples are shown in chapter 12 in the context of plugin_wiki, which uses MARKMIN extensively.

You can use markmin to generate HTML, LaTeX and PDF documents:

m = "Hello **world** [[link http://web2py.com]]"
from gluon.contrib.markmin.markmin2html import markmin2html
print markmin2html(m)
from gluon.contrib.markmin.markmin2latex import markmin2latex
print markmin2latex(m)
from gluon.contrib.markmin.markmin2pdf import markmin2pdf
print markmin2pdf(m) # requires pdflatex

(the MARKMIN helper is a shortcut for markmin2html)

Here is a basic syntax primer:

SOURCE OUTPUT
# title title
## section section
### subsection subsection
**bold** bold
''italic'' italic
``verbatim`` verbatim
http://google.com http://google.com
http://...<a href="http://...">http:...</a>
http://...png<img src="http://web2py.comhttp://...png" />
http://...mp3<audio src="http://...mp3"></audio>
http://...mp4<video src="http://...mp4"></video>
qr:http://...<a href="http://..."><img src="http://web2py.comqr code"/></a>
embed:http://...<iframe src="http://..."></iframe>
[[click me #myanchor]]click me
$$\int_a^b sin(x)dx$$
Simply including a link to an image, a videos or an audio files without markup result in the corresponding image, video or audio file beling included automatically (for audio and video it uses html <audio> and <video> tags).

Adding a link with the qr: prefix such as

qr:http://web2py.com

results in the corresponding QR code being embedded and linking the said URL.

Adding a link tith the embed: prefix such as

embed:http://www.youtube.com/embed/x1w8hKTJ2Co

results in the page being embedded, in this case a youtube video is embedded.

Images can also be embedded with the following syntax:

[[image-description http://.../image.png right 200px]]

Unordered lists with:

- one
- two
- three

Ordered lists with:

+ one
+ two
+ three

and tables with:

----------
 X | 0 | 0
 0 | X | 0
 0 | 0 | 1
----------

The MARKMIN syntax also supports blockquotes, HTML5 audio and video tags, image alignment, custom css, and it can be extended:

1.
MARKMIN("``abab``:custom", extra=dict(custom=lambda text: text.replace('a','c'))

generates

1.
'cbcb'

Custom blocks are delimited by ``...``:<key> and they are rendered by the function passed as value for the corresponding key in the extra dictionary argument of MARKMIN. Mind that the function may need to escape the output to prevent XSS.

OBJECT

Used to embed objects (for example, a flash player) in the HTML.

OBJECT
1.
2.
3.
>>> print OBJECT('<hello>', XML('<b>world</b>'),
>>>
_src='http://www.web2py.com')
<
object src="http://www.web2py.com">&lt;hello&gt;<b>world</b></object>
OL

It stands for Ordered List. The list should contain LI tags. OL arguments that are not LI objects are automatically enclosed in <li>...</li> tags.

OL
1.
2.
>>> print OL('<hello>', XML('<b>world</b>'), _class='test', _id=0)
<
ol id="0" class="test"><li>&lt;hello&gt;</li><li><b>world</b></li></ol>
ON

This is here for backward compatibility and it is simply an alias for True. It is used exclusively for checkboxes and deprecated since True is more Pythonic.

ON
1.
2.
>>> print INPUT(_type='checkbox', _name='test', _checked=ON)
<
input checked="checked" type="checkbox" name="test" />
OPTGROUP

Allows you to group multiple options in a SELECT and it is useful to customize the fields using CSS.

OPTGROUP
1.
2.
3.
4.
5.
6.
7.
8.
>>> print SELECT('a', OPTGROUP('b', 'c'))
<
select>
<
option value="a">a</option>
<
optgroup>
<
option value="b">b</option>
<
option value="c">c</option>
</
optgroup>
</
select>
OPTION

This should only be used as part of a SELECT/OPTION combination.

OPTION
1.
2.
>>> print OPTION('<hello>', XML('<b>world</b>'), _value='a')
<
option value="a">&lt;hello&gt;<b>world</b></option>

As in the case of INPUT, web2py make a distinction between "_value" (the value of the OPTION), and "value" (the current value of the enclosing select). If they are equal, the option is "selected".

selected
1.
2.
3.
4.
5.
>>> print SELECT('a', 'b', value='b'):
<
select>
<
option value="a">a</option>
<
option value="b" selected="selected">b</option>
</
select>
P
P

This is for tagging a paragraph.

1.
2.
>>> print P('<hello>', XML('<b>world</b>'), _class='test', _id=0)
<
p id="0" class="test">&lt;hello&gt;<b>world</b></p>
PRE
PRE

Generates a <pre>...</pre> tag for displaying pre-formatted text. The CODE helper is generally preferable for code listings.

1.
2.
>>> print PRE('<hello>', XML('<b>world</b>'), _class='test', _id=0)
<
pre id="0" class="test">&lt;hello&gt;<b>world</b></pre>
SCRIPT
SCRIPT

This is include or link a script, such as JavaScript. The content between the tags is rendered as an HTML comment, for the benefit of really old browsers.

1.
2.
3.
4.
>>> print SCRIPT('alert("hello world");', _language='javascript')
<
script language="javascript"><!--
alert("hello world");
//--></script>
SELECT
SELECT

Makes a <select>...</select> tag. This is used with the OPTION helper. Those SELECT arguments that are not OPTION objects are automatically converted to options.

1.
2.
3.
4.
5.
>>> print SELECT('<hello>', XML('<b>world</b>'), _class='test', _id=0)
<
select id="0" class="test">
<
option value="&lt;hello&gt;">&lt;hello&gt;</option>
<
option value="&lt;b&gt;world&lt;/b&gt;"><b>world</b></option>
</
select>
SPAN
SPAN

Similar to DIV but used to tag inline (rather than block) content.

1.
2.
>>> print SPAN('<hello>', XML('<b>world</b>'), _class='test', _id=0)
<
span id="0" class="test">&lt;hello&gt;<b>world</b></span>
STYLE
STYLE

Similar to script, but used to either include or link CSS code. Here the CSS is included:

1.
2.
3.
4.
>>> print STYLE(XML('body {color: white}'))
<
style><!--
body { color: white }
//--></
style>

and here it is linked:

1.
2.
3.
>>> print STYLE(_src='style.css')
<
style src="style.css"><!--
//--></
style>
TABLE, TR, TD

TABLE
TR
TD

These tags (along with the optional THEAD, TBODY and TFOOTER helpers) are used to build HTML tables.

1.
2.
>>> print TABLE(TR(TD('a'), TD('b')), TR(TD('c'), TD('d')))
<
table><tr><td>a</td><td>b</td></tr><tr><td>c</td><td>d</td></tr></table>
TR expects TD content; arguments that are not TD objects are converted automatically.
1.
2.
>>> print TABLE(TR('a', 'b'), TR('c', 'd'))
<
table><tr><td>a</td><td>b</td></tr><tr><td>c</td><td>d</td></tr></table>

It is easy to convert a Python array into an HTML table using Python's * function arguments notation, which maps list elements to positional function arguments.

Here, we will do it line by line:

1.
2.
3.
>>> table = [['a', 'b'], ['c', 'd']]
>>>
print TABLE(TR(*table[0]), TR(*table[1]))
<
table><tr><td>a</td><td>b</td></tr><tr><td>c</td><td>d</td></tr></table>

Here we do all lines at once:

1.
2.
3.
>>> table = [['a', 'b'], ['c', 'd']]
>>>
print TABLE(*[TR(*rows) for rows in table])
<
table><tr><td>a</td><td>b</td></tr><tr><td>c</td><td>d</td></tr></table>
TBODY
TBODY

This is used to tag rows contained in the table body, as opposed to header or footer rows. It is optional.

1.
2.
>>> print TBODY(TR('<hello>'), _class='test', _id=0)
<
tbody id="0" class="test"><tr><td>&lt;hello&gt;</td></tr></tbody>
TEXTAREA
TEXTAREA

This helper makes a <textarea>...</textarea> tag.

1.
2.
>>> print TEXTAREA('<hello>', XML('<b>world</b>'), _class='test')
<
textarea class="test" cols="40" rows="10">&lt;hello&gt;<b>world</b></textarea>

The only caveat is that its optional "value" overrides its content (inner HTML)

1.
2.
>>> print TEXTAREA(value="<hello world>", _class="test")
<
textarea class="test" cols="40" rows="10">&lt;hello world&gt;</textarea>
TFOOT
TFOOT

This is used to tag table footer rows.

1.
2.
>>> print TFOOT(TR(TD('<hello>')), _class='test', _id=0)
<
tfoot id="0" class="test"><tr><td>&lt;hello&gt;</td></tr></tfoot>
TH
TH

This is used instead of TD in table headers.

1.
2.
>>> print TH('<hello>', XML('<b>world</b>'), _class='test', _id=0)
<
th id="0" class="test">&lt;hello&gt;<b>world</b></th>
THEAD
THEAD

This is used to tag table header rows.

1.
2.
>>> print THEAD(TR(TH('<hello>')), _class='test', _id=0)
<
thead id="0" class="test"><tr><th>&lt;hello&gt;</th></tr></thead>
TITLE
TITLE

This is used to tag the title of a page in an HTML header.

1.
2.
>>> print TITLE('<hello>', XML('<b>world</b>'))
<
title>&lt;hello&gt;<b>world</b></title>
TR
TR

Tags a table row. It should be rendered inside a table and contain <td>...</td> tags. TR arguments that are not TD objects will be automatically converted.

1.
2.
>>> print TR('<hello>', XML('<b>world</b>'), _class='test', _id=0)
<
tr id="0" class="test"><td>&lt;hello&gt;</td><td><b>world</b></td></tr>
TT
TT

Tags text as typewriter (monospaced) text.

1.
2.
>>> print TT('<hello>', XML('<b>world</b>'), _class='test', _id=0)
<
tt id="0" class="test">&lt;hello&gt;<b>world</b></tt>
UL

Signifies an Unordered List and should contain LI items. If its content is not tagged as LI, UL does it automatically.

UL
1.
2.
>>> print UL('<hello>', XML('<b>world</b>'), _class='test', _id=0)
<
ul id="0" class="test"><li>&lt;hello&gt;</li><li><b>world</b></li></ul>
embed64

embed64(filename=None, file=None, data=None, extension='image/gif') encodes the provided (binary) data into base64.

filename: if provided, opens and reads this file in 'rb' mode. file: if provided, reads this file. data: if provided, uses the provided data.

embed64
xmlescape

xmlescape(data, quote=True) returns an escaped string of the provided data.

xmlescape
1.
2.
>>> print xmlescape('<hello>')
&
lt;hello&gt;

Custom helpers

TAG
TAG

Sometimes you need to generate custom XML tags. web2py provides TAG, a universal tag generator.

1.
{{=TAG.name('a', 'b', _c='d')}}

generates the following XML

1.
<name c="d">ab</name>

Arguments "a", "b", and "d" are automatically escaped; use the XML helper to suppress this behavior. Using TAG you can generate HTML/XML tags not already provided by the API. TAGs can be nested, and are serialized with str(). An equivalent syntax is:

1.
{{=TAG['name']('a', 'b', c='d')}}

If the TAG object is created with an empty name, it can be used to concatenate multiple strings and HTML helpers together without inserting them into a surrounding tag, but this use is deprecated. Use the CAT helper instead.

Notice that TAG is an object, and TAG.name or TAG['name'] is a function that returns a temporary helper class.

MENU
MENU

The MENU helper takes a list of lists or of tuples of the form of response.menu (as described in Chapter 4) and generates a tree-like structure using unordered lists representing the menu. For example:

1.
2.
3.
4.
5.
>>> print MENU([['One', False, 'link1'], ['Two', False, 'link2']])
<
ul class="web2py-menu web2py-menu-vertical">
<
li><a href="link1">One</a></li>
<
li><a href="link2">Two</a></li>
</
ul>

Each menu item can have a fourth argument that is a nested submenu (and so on recursively):

1.
2.
3.
4.
5.
6.
7.
8.
9.
>>> print MENU([['One', False, 'link1', [['Two', False, 'link2']]]])
<
ul class="web2py-menu web2py-menu-vertical">
<
li class="web2py-menu-expand">
<
a href="link1">One</a>
<
ul class="web2py-menu-vertical">
<
li><a href="link2">Two</a></li>
</
ul>
</
li>
</
ul>

The MENU helper takes the following optional arguments:

mobile

MENU takes an optional argument mobile. When set to True instead of building a recursive UL menu structure it returns a SELECT dropdown with all the menu options and a onchange attribute that redirects to the page corresponding to the selected option. This is designed an an alterantive menu representation that increases usability on small mobile devices such as phones.

Normally the menu is used in a layout with the following syntax:

{{=MENU(response.menu, mobile=request.user_agent().is_mobile)}}

In this way a mobile device is automatically detected and the menu is rendered accordingly.

BEAUTIFY

BEAUTIFY is used to build HTML representations of compound objects, including lists, tuples and dictionaries:

1.
{{=BEAUTIFY({"a": ["hello", XML("world")], "b": (1, 2)})}}
BEAUTIFY returns an XML-like object serializable to XML, with a nice looking representation of its constructor argument. In this case, the XML representation of:
1.
{"a": ["hello", XML("world")], "b": (1, 2)}

will render as:

1.
2.
3.
4.
<table>
<
tr><td>a</td><td>:</td><td>hello<br />world</td></tr>
<
tr><td>b</td><td>:</td><td>1<br />2</td></tr>
</
table>

Server-side DOM and parsing

element
elements

elements

The DIV helper and all derived helpers provide the search methods element and elements.

element returns the first child element matching a specified condition (or None if no match).

elements returns a list of all matching children.

element and elements use the same syntax to specify the matching condition, which allows for three possibilities that can be mixed and matched: jQuery-like expressions, match by exact attribute value, match using regular expressions.

Here is a simple example:

1.
2.
3.
4.
5.
>>> a = DIV(DIV(DIV('a', _id='target',_class='abc')))
>>>
d = a.elements('div#target')
>>>
d[0][0] = 'changed'
>>> print a
<div><div><div id="target" class="abc">changed</div></div></div>

The un-named argument of elements is a string, which may contain: the name of a tag, the id of a tag preceded by a pound symbol, the class preceded by a dot, the explicit value of an attribute in square brackets.

Here are 4 equivalent ways to search the previous tag by id:

1.
2.
3.
4.
>>> d = a.elements('#target')
>>>
d = a.elements('div#target')
>>>
d = a.elements('div[id=target]')
>>>
d = a.elements('div',_id='target')

Here are 4 equivalent ways to search the previous tag by class:

1.
2.
3.
4.
>>> d = a.elements('.abc')
>>>
d = a.elements('div.abc')
>>>
d = a.elements('div[class=abc]')
>>>
d = a.elements('div',_class='abc')

Any attribute can be used to locate an element (not just id and class), including multiple attributes (the function element can take multiple named arguments), but only the first matching element will be returned.

Using the jQuery syntax "div#target" it is possible to specify multiple search criteria separated by a space:

1.
2.
>>> a = DIV(SPAN('a', _id='t1'),DIV('b',_class='c2'))
>>>
d = a.elements('span#t1, div#c2')

or equivalently

1.
2.
>>> a = DIV(SPAN('a', _id='t1'),DIV('b',_class='c2'))
>>>
d = a.elements('span#t1','div#c2')

If the value of an attribute is specified using a name argument, it can be a string or a regular expression:

1.
2.
>>> a = DIV(SPAN('a', _id='test123'), DIV('b',_class='c2'))
>>>
d = a.elements('span', _id=re.compile('test\d{3}')

A special named argument of the DIV (and derived) helpers is find. It can be used to specify a search value or a search regular expression in the text content of the tag. For example:

1.
2.
3.
4.
>>> a = DIV(SPAN('abcde'), DIV('fghij'))
>>>
d = a.elements(find='bcd')
>>>
print d[0]
<
span>abcde</span>

or

1.
2.
3.
4.
>>> a = DIV(SPAN('abcde'), DIV('fghij'))
>>>
d = a.elements(find=re.compile('fg\w{3}'))
>>>
print d[0]
<
div>fghij</div>

components

Here's an example of listing all elements in an html string:

1.
2.
html = TAG('<a>xxx</a><b>yyy</b>')
for item in html.components: print item

parent

parent returns the parent of the current element.

1.
2.
3.
4.