Python Final

CYBER SECURITY INFORMATION TECHNOLOGY

AppSec
CRITICAL INFRASTRUCTURE HOMELAND SECURITY MULTI-USER NETWORK CYBER SECURITY INFORMATION TECHNOLOGY CRITICAL INFRASTRUCTURE

(By Rich Helton)
Python for
Security Professionals (September 21, 2011)
(with some Droid and Django)

State of Colorado
Office of Cyber Security

State of Colorado Office of Cyber Security

Class Introduction
CYBER SECURITY INFORMATION TECHNOLOGY CRITICAL INFRASTRUCTURE HOMELAND SECURITY MULTI-USER NETWORK CYBER SECURITY INFORMATION TECHNOLOGY CRITICAL INFRASTRUCTURE

  My information, rich.helton@state.co.us

  A good reference for an online Python class is
http://code.google.com/edu/languages/google-python-class/
introduction.html , PyQuick Python Class from Google University.
  Another place for reference Python Documentation,
http://www.python.org/doc/


Python History

  From
http://en.wikipedia.org/wiki/Python_(programming_language) ,
  Python is an interpreted, general-purpose high-level programming
language whose design philosophy emphasizes code readability.
  The reference implementation of Python (CPython) is free and open
source software and has a community-based development model, as
do nearly all of its alternative implementations.
  Python interpreters are available for many operating systems, and
Python programs can be packaged into stand-alone executable code
for many systems using various tools.
  Python’s implementation was started in December 1989 by Guido
van Rossum in the Netherlands as a successor to the ABD
programming language.


Installing Python

  Python can be downloaded and installed from
http://www.python.org/getit/


Just my 2 Cents,

  Referencing others, Python does not define types ahead of time,
but uses declaration at runtime. Many say the variable names in
Python have to be watched more closely due to this interpreted
runtime.
  You need to be prepared to create modules and break apart lines
more often to keep readability more apparent, while a statement in
Java code could easily occupy three lines of text, Python works with
line breaks and indentation and cannot easily occupy multiple lines for
the one statement. This concept ensures Python as a more compact
language and the need to occupy less space.
  This in turn makes Python able to get a lot more done with fewer
lines of code and to be great at utilities and tools. Python is concise.
  Keep it simple,


Other Advantages

  Python does not require as much typing as Java and integrates
more into the Operating System as a scripting language.
  While Java has many more frameworks than Python, Python does
have a Spring Framework (Middleware) and many Web Frameworks.
  I like to work with Python for admin and 3D graphics integration:
  If you do graphics programming, Python is a must have. There are packages
like Maya, that uses Python plugins, Blender 3D, and Disney’s original Panda3D
application is based on Python.
  If you manage Linux systems and do network and system security, Python is a
language that many scanning tools are based on.
  Python is a great language for incorporating modules from other languages and
applications.
  For a list Python software, see
http://en.wikipedia.org/wiki/List_of_Python_software


Python Software


Java vs. Python

  A simple “Hello World” (Python is concise, see left pane):


Panda3D (Free Python 3D Engine)

  Python can be powerful, see Disney’s game engine,
http://www.panda3d.org/


Panda3D (Free Python 3D Engine)

  Panda3D Boxing Program Sample, nothing but Python.


Pygame

  One of the most well known frameworks regarding python is
pygames, found at http://www.pygame.org/ and information found at
http://en.wikipedia.org/wiki/Pygame
  Pygame has been around 2004.
  It has many tutorials and editors like a Tile and Level Editor, and
even a Map Maker:


Pygame Aliens game


Pygame

  Pygame has a subset for the Android, see
http://pygame.renpy.org/ .
  To learn the code for Sprite animation for Aliens, simply go through
the code at
http://svn.pythonmac.org/py2app/py2app/trunk/examples/pygame/
aliens.py


My First Python Program

  Many of programs will be done in Windows and by installing
python. Linux can be used as well and python normally comes
standard in Linux.


My First Python Program (with Error)


Python, not just a Calculator

  Python can perform complex calculations with little effort. Don’t
forget to use numbers instead of literals, and note the # (comment
symbols):


Creating a py file

  Python files normally end with a “.py” extension.
  Using a Open Source editor, you can write a Python program, like
http://geanyportable.org/ or http://www.pspad.com/ .
  We create prog1.py:


Running a py file

  We can run the “py” file from the IDLE GUI shown as the interpreter
to interactively run it.
  Or at the command line:


Running a py file

  Other editors, like http://netbeans.org , a Java editor, have plugins
to run Python files. See also http://wiki.netbeans.org/Python .
  A plugin for Python will have to be installed:


Running a py file

  Running in NetBeans:


Debugging a py file

  Debugging in NetBeans, here we are line by line debugging and
printing out the www.google.com webpage in the output window :


Debugging a py file

  Let’s break down the code:


Debugging a py file
(Reading a Google URL)

  We can set the request name as a variable. The variable does not
require a type.


Using it across the WWW

  In this example, we grab the page to www.google.com and read the
output.
  We can now start to use this as a web crawler. A Web crawler will
grab the URL from the site and walk down it and then grab the next
URL, and walk down it, in many cases copying the pages into a local
directory to capture the web site structure.
  Some even come with Linux operating systems, like wget.
  Building your own crawler allows you to test other items as well,
such as putting a ‘ or “ in the forms to test for SQL Injection.


• Wget
• (Open Source Web downloaders)

  Open Source utilities, like wget, or VisualWget
http://www.bebits.com/app/299 , provide a means to download
all available URL’s on a Web Site:


• Wget
• (Open Source Web downloaders)

  Results:


Python Web Crawlers

  I will discuss the Python implementations more at a later time, but
here are some links to some Python Web Crawlers.
  There are many sites that discuss Python Web Crawlers, the
benefit is that you can add to look for specific patterns:
 Simple Web Crawler
http://code.activestate.com/recipes/576551-simple-web-crawler/
  http://ms4py.org/2010/04/27/python-search-engine-crawler-part-1/
  http://blog.theanti9.com/2009/02/14/python-web-crawler-in-less-than-50-lines/
 There are many sites that discuss Python ping.py implementation
that I will discuss in the network section, that may help in network
mapping:
  http://stackoverflow.com/questions/316866/ping-a-site-in-python
  http://www.g-loaded.eu/2009/10/30/python-ping/



Some Python Tools


PyLint

  PyLint is a static source analysis tool, available by command line
or integrated into tools like PyDev and frameworks like Django
  See http://en.wikipedia.org/wiki/Pylint
  After downloading the PyLint package and unzipping it from
http://pypi.python.org/pypi/pylint#downloads , run the “python setup.py
install” in its root package to install it to the local Python application.
  It will install it under the local PythonScripts directory, so make
sure that this directory is in the executable Path environment.


Running PyLint

  For our “test.py” file, we simply run “pylint test.py”:


PyDoc

  The pydoc module automatically generates documentation from
the Python modules. It can be presented as text in the console,
server to a Web browser, or saved to HTML files.
http://docs.python.org/library/pydoc.html
  Running “python pydoc.py sys” from the PythonLib directory gives
us info on the sys module:


PyDoc

  Running “python pydoc.py –g” will bring up the Tkinter GUI
interface to search and a button to load a browser with all the
modules:


PyUnit

  This is the Python version of Junit, http://pyunit.sourceforge.net/ ,
which is the standard unit testing framework.
  Download it from this website and install it with the “python setup.py
install” cmd.


PyUnit

  PyUnit makes use of the “unittest” module from Python, so it must
be installed. See http://pyunit.sourceforge.net/unittest.html
  The Python file will need an “import unittest” self using functions to
design a unit test. See http://docs.python.org/library/unittest.html


PyUnit

  Here we have “ArithTest” that will use “self.fail” functions to test the
cases:



Creating a Python
Executable


Download the py2exe.org

  Download the correct version from http://www.py2exe.org to match
the Python interpreter installed, i.e. 2.7.
  In this case it will be installed in the 2.7 directory….


Quick test

  Let’s cd to the download sample directory and run the simple
sample. The directory can be viewed from the installation dialog.
  The test will show an error. This is because there are multiple
Python versions and that’s why is didn’t compile. Panda3D was also
installed and installed 2.6.5, OOps:


To Compensate (Multiple Pythons)

  When applications are installed with Python support, it may install
another copy of Python.
  You can de-install down to one copy, and use a copy of Python that
is supported by all the local Python applications, likely not the most
current, or start compiling directly from the local directory.
  In this case, we will call the correct python directly and copy the
simple samples over to the python directory.
  Then we will run the local “python setup.py py2exe”. This setup.py
came with the sample and defines to compile a hello.py.
  There will be several pages of compile information ending with:


Compiled

  A “dist” directory will be created from the samples with the compiled
python scripts.
  The hello.py is now hello.exe:


Non-Windows Multiple Pythons

  You may notice that many Python files may have the first line with
something like “#!/usr/bin/python”. This means to use the python
under “/usr/bin” in Linux or MAC OSX.
  You may even see “#!/usr/bin/env python2.6” that simply means to
use the first Python 2.6 binary found in the user’s environment.
  Depending on what you put in this first line will control the Python
binary to use, if not used, it will find the default one in the current
environment. See
http://stackoverflow.com/questions/2429511/why-do-people-write-usr-
bin-env-python-on-the-first-line-of-a-python-script



LET’S BREAK DOWN THE
PROGRAM


Components of a program…

  Comments
  Reserved words
  Statements
  Blocks
  Classes
  Methods
  The main method


Comments

  Comments are used to communicate the purpose of parts of the
program in the code.
  Comments are not executed as part of the code and are ignored by
the compiler.
  Using the (#) double slashes will tell the compiler to ignore the
entire line.
  Using the (“ “ “) triple quote multiline comment to comment code it
meets the ending ending triple quote (“ “ “).


Reserved Words

  Reserved words, or keywords, are words with special meaning that
define how the program should behave in the compiler.
  These are words will explain to the compiler how to construct the
program.
  Being reserved to the compiler, these words cannot be duplicated
anywhere else in the code as class names, variable names, or
method names.
  Words in the program, such as “assert” tells the compiler to asset
that a statement is true or it will return an exception.


Reserved Words

  Python reserved words:


Conditionals

  Conditionals, if/else clauses, are probably the most important
construct in programming.
  It checks to see if a condition is true and executes its block,
otherwise it may execute a “else” if set.
http://docs.python.org/tutorial/controlflow.html


For Loop

  A “For” loop will increment through a specified length. Length can
be used instead of the integer.
  Remember to indent the code within conditions, loops, and
definitions, because the indentation defines the code within the
block.


For Loop 2
  Let’s create something more useful, like printing out the OS
environment


Operating System Commands
  Python has many Operating System (os) commands that are part of
the Python os module, see http://effbot.org/librarybook/os.htm
and http://docs.python.org/library/os.html
  These commands can create and remove files, directories, get
current environment variables, get information on files and
directories, change permissions on files and directories, start
programs, end programs, start services and daemons and mush
more.
  We will look at it more in a bit, back to loops.


While Loop
  A While loop will continue until a condition is met, this is example
is in cygwin, like Linux, so it has a header “#!/usr/bin/python” :


Statements

  A statement is line for a singe unit of work.
  A line in Python is ended just at the end of line in the file.


Blocks

  A block is a group of actions that is normally defined in a method,
condition, exception, loop or other area construct.
  A block is started with the “:” in the section, defined by indentation
until another block is encountered.


Main method

  Python with multiple functions need to define a “main” method, it
defines the method in the “if __name__ == "__main__":” block.


Object Oriented Development
  Python supports Object Oriented Programming (OOP), here is an
example, see
http://www.zugiart.com/notes/programming/python/ :


What is a class?

  A class is a definition of a group of methods, variables, state and
behavior.
  In the sample, the class is MyClass.
  As MyClass it can contain variables, methods, state and behavior
that are defined. In this case it simply prints “helloWorld” with an
argument.
  The idea of a class is that is can define a car, a catalog, and another
group of a construct.
  The class can create subgroups, like a car class can have sub classes
of itself like “blue car” or “compact car” to define the construct in
more detail.
  When a class is created and started, called instantiation, it becomes
an object. A class can create many objects.


Creating a module
  Python statements are not very long compared to other languages,
and by that I simply mean that while I could easily write a
statement that takes 3 lines of code, Python delaminates on
indentation and end of line.
  To compact the code, modules are used to break the code into
different libraries, just like functions and classes are used to break
the code up into different blocks in the file.
  See http://docs.python.org/tutorial/modules.html


Creating a module
  Let’s create a Fibonacci numbers module with a “fib” function:


Creating a module
  Let’s call the fib.py module and “fib” function from a myFibo.py file
using the “import” keyword:



OS Module


OS Module
  We briefly touched the os module in the last chapter. We will walk
through a few programs (Getting File info, this was Windows):


OS Module
  Here we copy a file and give it the same info as the original:


OS Module
  Changing to a subdirectory called samples and back again:



Lists, Sets and Tuples


Lists
  A list in Python is an ordered group of items or elements.
http://en.wikibooks.org/wiki/Python_Programming/Lists
  They use “ ( … )” parenthesis and their data can be changed.


Tuples
  Tuples in Python are much like lists, except they are immutable,
meaning once it is created, it is never changed. See
http://en.wikibooks.org/wiki/Python_Programming/Tuples
  Notice parenthesis are used instead of brackets.


Sets
  Sets in Python are a type of mathematical collection that supports
set operations like unions, intersections, differences and more. See
http://en.wikibooks.org/wiki/Python_Programming/Sets



Types of Web Hacking


• Web Attacks are the most common
• (from the 2010 ArcSight survey)


• There are many hacks….

  And more are discovered every day. For 2010, the Open Web
Application Security Project (OWASP) published the top ten web
hacks,
http://www.owasp.org/index.php/Category:OWASP_Top_Ten_Project :
1)  Injection Flaws
2)  Cross Site Scripting (XSS)
3)  Broken Authentication and Session Management
4)  Insecure Direct Object Reference
5)  Cross Site Request Forgery (CSRF)
6)  Security Misconfiguration
7)  Insecure Cryptographic Storage
8)  Failure to Restrict URL Access
9)  Insufficient Transport Layer Protection
10)  Unvalidated Redirects and Forwards

• SANs 2010 Top Cyber Security Risks


There are many Checklists….

  Here are some checklists to help with the principles:
  The ASP.NET Security Checklist
http://msdn.microsoft.com/en-us/library/ff648269.aspx
  WebAppSec Excel checklist
http://img.a4apphack.com/dl/appsecchck-checklist.zip
  SANs reading Web Security Checklist
http://www.sans.org/reading_room/whitepapers/
securecode/security-checklist-web-application-
design_1389
  The Open Web Application Security Project Application
checklist is
http://www.sans.org/reading_room/whitepapers/
securecode/security-checklist-web-application-
design_1389


Checklists Principles seem the same…

  All the checklists, regardless of programming languages, have
many of the same principles:
  Find and validate all input. This includes URL’s,
JavaScript's, links, username and passwords, and
especially any field calling a database.
  Never trust data in files, the network or database to be
secure. Encrypt anything important, passwords, SSN’s,
configurations.
  Never trust the source, be it customer or a service.
Authenticate, Authorize and validate.
  Whenever a abnormal behavior occurs, error check and
log.
  Keep testing, as people from all skills will be testing
anything on line and may try common threats.


SQL Injection
(Most common Injection
Flaw)


Intro to SQL Injection…

  Many web pages communicate directly to a backend database for
processing.
  For example, a username and password is asked for on the Web
page and the web page will pass it to the database to validate the
information.
  Some applications will not validate the field adequately before
passing it to the database, and the database will process whatever it
will receive.
  Hackers will pass SQL commands directly to the database, and in
some cases tables like “passwords” are returned because the SQL
commands are not being filtered adequately.
  SQL may return errors in the web page that even lists the correct
tables to query so that the hacker may make more accurate attempts
to get data.

SQL Injection

  SQL Injection is the ability to inject malicious SQL commands
into the backend code.
  For example:
SELECT * FROM users WHERE username = ‘USRTEXT '
AND password = ‘PASSTEXT’
  Passing ' OR 1=1-- in the USRTEXT field generates:
SELECT * FROM users WHERE username = ‘’ OR 1=1 -- '
AND password = ‘PASSTEXT’

  The OR 1=1 returns true and the rest is commented out


• Live sites…

  There are live web sites that simulate banks and other businesses to
practice web hacking and test hacking tools against. They include:
  Spi Dynamics - http://zero.webappsecurity.com/
  Cenzic - http://crackme.cenzic.com/Kelev/view/home.php
  WatchFire - http://demo.testfire.net/
  HackThisSite - http://www.hackthissite.org/
  NTO - http://hackme.ntobjectives.com/
  Accunetix - http://testaspnet.acunetix.com/login.aspx


SQL Inj with Python

  We are going to http://demo.testfire.net/bank/login.aspx , and
looking at the site, we will see that Username is the field “uid” and
Password is the field “passw”:


SQL Inj with Python
  Running the following script will put a ‘ (tick) in uid and ‘ (tick) in
passw:


SQL Inj with Python
  The results, OleDB errors


SQL Inj with Python
  If we were to load the results in the browser:


SQL Inj with Python
  The OleDB starts to give us enough information to start looking at
the database to see if we can get through. We can see that the field
in the database is “username”.
  While Python simulates what we can do manually, automating the
process allows us to start looping though the website to find more
information faster.



Jython


Jython

  If you installed NetBeans correctly, you would notice that Jython
was installed as well.
  Jython, http://en.wikipedia.org/wiki/Jython , is Python for the Java
platform.
  The language can be found at http://www.jython.org/ .
  Python can run in a Java Virtual Machine (JVM) and call Java
Frameworks that will extend Python. Therefore, Python can run from
any machine that will run Java.
  This also means that Java can be run from the Python Language.
  Jython can be downloaded and installed from
http://www.jython.org/downloads.html
  Installation Instructions are found at
http://wiki.python.org/jython/InstallationInstructions


Jython

  After installing Jython by running the installation JAR, we run the
jython.bat (in XP) under the jythonbin directory.
  It gives us a command prompt similar to Python, and now we can
run Java frameworks from the Python language, here we will call the
Java Util package’s Date class:



Strings


Strings

  A String is just a form of representing text.
  A link for various functions in types, including strings is
http://docs.python.org/library/stdtypes.html .
  We can use the python command line interpreter to fill in a string
variable “fred”:


Cases

  We can check and set fred to a lower case:

 Now upper:



Regex


Regex

  Regex is short for Regular Expression, see
http://en.wikipedia.org/wiki/Regular_expression
  Regex is simply a form pattern matching to strings or text, and a
method of strings for finding these patterns.
  A large library of various Regex expressions can be found at
http://www.regxlib.com/
  A pattern for finding SSNs can be found at Regex Lib for the
following:


Regex
  Let’s test the previous SSN Regex. It will show a “matched” string if
it matches (It matches):

  Also see
http://www.tutorialspoint.com/python/python_reg_expressions.htm

Regex Cheatsheet
  There are many Cheat Sheets to help people learning Regex, one
of my favorites is at http://regexlib.com/CheatSheet.aspx :



Secret Writing
(Encryption)


Who’s seeing your data?

  Data at rest and in transient can be at risk to prying eyes.
  When a system is in production, and especially on the Internet,
there is no guarantee that you know who is watching the data
transmitted between the user and the server. This may also apply to
the Local Area Network as well.
  Never take it for granted that access cannot be broken.
  Always, use common algorithms that come with Python, Java or
C#.
  Common algorithms are tested well and are vetted by millions.
  Keep the keys as secure as the data, because they can unlock the
data.
  Homemade encryptions algorithms may end up costing more than
standard encryptions because the algorithm may be broken.


One-way Hash Algorithms

  There are two common types of one-way hash algorithms,
Message Digest 5 (md5), and Secure Hash Algorithm 1
(sha1).
  The one-way hash generates a fixed size hash from some
given data of any size.
  The data cannot be reversed engineered from the hash,
hence one-way.
  The same data generates the same hash sum.
  Different data generates different hash sums.
(Note: In rare cases, collisions, different data generates the
same sum).


Md5

  Message Digest 5 (md5) will take data input and generate
128 bit hash sum.
  The 128 bit hash sum can be used to ensure if there has
been tampering of data or a file.
  A common comparison is to store hashes in a table instead
of the original password. Instead of checking the
password, compare the hash of the password, so that the
password does not have to be stored, and only a hash is
used to check the original password. Therefore the
original password is never stored or seen, only the hash.


Md5

  Notice that the same values being hashed produce the
same hash:


Sha1

  Secure Hashing Algorithm 1 (Sha1) will take any size byte
array and produce a 160 bit hash sum, sometimes called a
message digest.
  Other SHA’s are SHA224 (224 bits), SHA256 (256 bits),
SHA384 (384 bits), and SHA512 (512 bits), each one
denoting the size in bits of the message digest.


Sha1

  Only changed 2 lines of code from Md5:


AES

  The National Security Agency (NSA) updated their algorithm in
2001 to the Advanced Encryption Algorithm (AES) for Top Secret
information from the Data Encryption Standard in 1975.
  The Rijndael algorithm was selected, developed by two Belgian
cryptographers, Joan Daemen and Vincent Rijmen.
  The NIST adapted the variable key space into 128, 192, or 256 bits
as FIPS 197 and called it AES.
  AES is a symmetric key algorithm, meaning that the same key is
used to both encrypt and decrypt.

 To use AES algorithm in Python, a library like the PyCrypto module
has to be installed, one can be found at
http://www.voidspace.org.uk/python/modules.shtml#pycrypto


Python Crypto Modules

  There are several Crypto modules for Python, PyCrypto seems to
be very popular and supports many algorithms, but for a 2010
comparison of many of the popular modules, please see
http://mikeivanov.com/pc/python-crypto.pdf


AES Python Code
 Initialing an Initialization Vector for AES, and the password , Cipher
Mode, and Salt. They must match on both sides. :


RSA

  The most important Asymmetric algorithm to understand is the
Rivest-Shamir-Adleman (RSA). So named after the MIT
mathematician inventors in 1978.
  The Asymmetric algorithm can generate key pairs, one private key
for encrypting, and its pair is handed out for decryption to more
people, the public key.
  The key pair are formulated from a pair of prime numbers using a
modulus equation that become linked to each other, therefore only
pieces of the formula need to be exchanged and not the complete
keys, and only their side that they use is calculated.


RSA Keys, a simple example

1) Choose two prime numbers p and q.
P = 61 and q = 53.
2) Compute n = pq, n = 61 * 53 = 3233.
3) Compute the totient φ(n) = (p – 1)(q – 1) =
(61 -1) * (53 – 1) = 3120.
4) Choose a coprime e (like 17) that is not a divisor of the totient.
5) Compute d such that e*d mod φ(n) = 1.
17 * 2753 (d) = 46801, 46801 mod 3120 = 1.
public key = (e, n) = (17, 3233)
private key = (d, n) = ( 2753, 3233)


RSA Keys, a simple encrypt/decrypt

public key = (e, n) = (17, 3233)
private key = (d, n) = ( 2753, 3233)

To compute the ciphertext we use
C = Pe (mod n).
For example, P = 65 and is the letter ‘H’.
C = 2790 = 6517 mod 3233.

Back to Plaintext, P = Cd mod n.
P = 65 = 27902753 mod 3233. Which returns 65 for ‘H’.


RSA

  Using a RSA Python program from
http://www.prasannatech.net/2008/09/rsa-encryption-algorithm-large-
integers.html, we can perform these complex calculations. The
program selected e = 7 as the quotient (Python helps with complex
calculations):


RSA
  Here’s a RSA example with PyCrypto, similar to the AES sample:



Files


Files

  Reading and writing files is a basic fundamental of a programming
language.
  Many of the functions for File I/O will simply be a read or write
function using a File class.
  Some of this can be referenced from the Python documentation at
http://docs.python.org/tutorial/inputoutput.html


Writing a File


Reading a File


Directory

  Python is really handy for doing directory structure traversal.
  Some of the available file and directory routines can be seen at
http://docs.python.org/release/2.5.2/lib/os-file-dir.html
  Here’s a quick program to list directories with an argument of the
directory, I used “.” for the current one, (used Notepad++) :



XML


Extensible Markup Language (XML)

  XML is a specification for creating custom markup languages.
  An example of a language that XML can create is Hypertext Markup
Language (HTML). HTML is the markup language for displaying
web pages.
  XML is very flexible because it uses tags to describe data elements:
<book>This is a book... </book>
  An XML file is commonly use to transfer language elements over
HTTP in the form of Web Services or AJAX.
  The XML file must also contain the proper header information to
state that the XML version and encoding scheme:
<?xml version="1.0" encoding="UTF-8"?>
There are several types of parsers for XML but the basic types are
Document Object Model (DOM) and Simple API for XML (SAX).


HTMLParser

  HTML is one of the languages generated from XML. In other
words, HTML is a meta-language and a subset of XML.
  XML creates the HTML grammar from its specific Document Type
Definition (DTD)
  Python includes a HTMLParser, see
http://docs.python.org/library/htmlparser.html
  Let’s parse (Looks like XML):


HTMLParser

  This is why Python is powerful, not a lot of code:


Running HTMLParser

  Running it, the program prints the start and end tags:


SAX

  SAX, Simple API for XML, see
http://docs.python.org/library/xml.sax.html, python will use the
xml.sax module.
  It basically opens the XML file with an XML reader, and passes the
file to a handler.
  The handler will traverse the XML file, parsing the tags for specific
input until it finishes with the file.
  SAX operates on the individual tags where DOM operates on the
document as a whole,
http://en.wikipedia.org/wiki/Simple_API_for_XML


SAX

  Looking at a library.xml:


SAX

  The file for parsing out the book and author tags:


SAX

  Running the file:


DOM

  Document Object Model (DOM) is the XML parsing technique used
by most browsers, see
http://en.wikipedia.org/wiki/Document_Object_Model
  The DOM parser uses a Document Type Definition file that defines
the elements in the XML file that makes up the markup language.
  The program loads the entire file into a structured hierarchy,
usually storing elements in tree collection.
  DOM requires more RAM to load the files in the memory all at
once.
  Python makes use of minidom, a lightweight DOM implementation
using the “xml.dom.minidom” module, see
http://docs.python.org/library/xml.dom.minidom.html


DOM

  Let’s parse out “books.xml”:


DOM

  We will use “pprint”, Data pretty printer to print the data, see
http://docs.python.org/library/pprint.html .
  The code to parse it may look something like:


DOM

  Running the program will produce the following, which is a
printout of the different elements:



Databases


Databases
  Writing and reading from a database is a rudimentary skill of any

programming language, and Python works with most common
databases, see http://wiki.python.org/moin/DatabaseProgramming/
and http://wiki.python.org/moin/DatabaseInterfaces


SQLite3

  Whenever possible, I like to use SQLite3,
http://en.wikipedia.org/wiki/SQLite, simply because there is a lot of
iPhone and Android support if I want to make my App mobile.
  The Python documentation for Sqlite3 can be found at
http://docs.python.org/py3k/library/sqlite3.html


SQLite3

  Here, we create a sample.db, add text, like one, and interger, like 1,
and read the rows:


SQLite3

  It did create a sample.db file that is the database:


SQLite3

  Just reading the rows:


MySQL

  We are going to install MySQL and the Python module for MySQL,
http://www.codegood.com/archives/4 .
  A free Open Source database that can run as a service and is more
relational for production use is the MySQL database. See
http://en.wikipedia.org/wiki/MySQL .
  Install MySQL from http://www.mysql.com/downloads/mysql/
  Here is an example to create the myTestAppDB:
  mysql> create database myTestAppDB character set utf8;
  Here is an example to create user “myUser” password “XXXXXXXX”
•  mysql> CREATE USER ‘myUser’@’localhost’ IDENTIFIED BY
‘XXXXXXXX’;
•  mysql> GRANT ALL PRIVILEGES ON *.* TO ‘jirauser’@’localhost’
WITH GRANT OPTION;


MySQL

  To test to see if the MySQLdb module is installed, we will check the
version of MySQL. An error will return if the library is not set up
correctly.


MySQL

  Now we are going to create an animal tabl and 4 rows of animals,
see http://www.kitebird.com/articles/pydbapi.html .



Network Programming


Network Programming

  A good Tutorial on Network programming in Python can be found
http://heather.cs.ucdavis.edu/~matloff/Python/PyNet.pdf .
  Python uses sockets, very similar in convention to Windows and
Java socket programming.
  Socket programming involves first creating a Socket server, that will
have a base port number to listen on and accept incoming
connections. This is a connection oriented socket, i.e. TCP.
  A incoming connection is generated from the client that will create a
socket, connect to the Server’s host port number and address, and
send information to the server.
  Sockets were originally derived from Berkeley sockets,
http://en.wikipedia.org/wiki/Berkeley_sockets .


Python Final

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Python Final

Similaire à Python Final (20)

Plus de Rich Helton

Plus de Rich Helton (20)

Python Final