Benefits

The major benefit of OSP for VoIP carriers is it provides a single, highly secure mechanism for managing diverse VoIP networks. OSP provides a common interface between VoIP networks and the operations and billing support systems (OSS/BSS) used to manage VoIP networks. By using a global VoIP to OSS/BSS interface, carriers are now enabled to build and manage multi-vendor, multi-protocol networks without impacting their central routing and billing operations. The benefits are greater flexibility, vendor independence and lower operating costs.

Features

• Common peering mechanism for diverse VoIP networks.

• Digitally signed peering tokens enable secure, ubiquitous peer to peer communications

• Interdomain access control much more secure and scalable than IP access lists.

• Common, standard Call Detail Record (CDR) for diverse VoIP networks.

• Real time collection of CDRs from both source and destination networks.

• CDR collection can be encrypted using SSL ensuring secure data collection.

• High CDR integrity minimizes settlement disputes.

• OSP messages are XML based which are easy to read, manipulate and extend.

• Supports both H.323 networks, SIP networks and can support other IP transaction models.

• Broad vendor support.

Overview of OSP

The OSP standard defines fourteen messages which can be used to convey peering authorization and capabilities. This document focuses on the six messages most commonly used to manage VoIP peering. These six messages typically occur between an OSP client such as a VoIP gateway, SIP Proxy or softswitch and an OSP Server. These messages, written in italics, are:

1. AuthorizationRequestThe message sent from an OSP client (i.e. source gateway) to the OSP Server requesting one or more routes and authorization to a destination gateway that can complete the call to the dialed number.

2. AuthorizationResponse:The response from the OSP Server to the AuthorizationRequest from a source gateway. This message includes one or more IP addresses of destination devices that can complete the VoIP call to the dialed number. Each destination IP address is accompanied with an authorization token digitally signed by the OSP Server. The authorization token is included in the call set-up request from the source device to the destination device. The destination device validates the token before accepting the call set-up request from the source device.

3. UsageIndication: This message is the OSP defined message for reporting usage. The source and destination devices automatically send a UsageIndication message to the OSP Server when the call is completed.

4. UsageConfirmation: The OSP Server responds to each UsageIndication message with a UsageConfirmation message. A source or destination device will continue to send a UsageIndication message for a call until it receives a UsageConfirmation from the OSP Server.

5. CapabilitiesIndication: This versatile message is sent from an OSP client to an OSP Server. The OSP client may use the capabilities exchange message to transmit the following information to an OSP Server:

a. Serial number and customerID (can be used for self provisioning)

b. Services requested from the OSP Server (i.e. routing, accounting or both)

c. OSP version supported

d. Available bandwidth and channel resources (a client can indicate to an OSP Server that it should be added or removed from routing based on its available resources.)

6. CapabilitiesConfirmation: The OSP Server responds to each CapabilitiesIndication message with a CapabilitiesConfirmation message.

Simple OSP Call Scenario

A single OSP Server authorizes a peer to peer call between two voice over IP gateways. The originating (source) peer makes an OSP AuthorizationRequest to the OSP Server for the IP address of a terminating (destination) peer that can complete the call to the dialed number. The OSP AuthorizationResponse includes the IP address of the terminating (destination) peer and a peering authorization token digitally signed by the OSP Server.

The originating (source) peer includes the peering authorization token in the call set-up to the terminating (destination) peer. An peering authorization token can be included in a Q.931 call set-up message or in a SIP INVITE message. The terminating gateway recognizes the authorization token in the call set-up, validates that the authorization token was digitally signed by a trusted peering authority.

As shown in the illustration below, when the call ends, the originating and terminating gateways send their UsageIndication messages to the OSP Server. The OSP Server responds by sending a UsageConfirmation message (not shown) to the originating and terminating gateways.

• PL/SQL Debugger, at least according to the specs. I couldn’t get it going. The menu showed the icon, but was disabled.
• SQL Worksheet with syntax highlighting. Tab Pages provide additional information such as Explain Plan and a Log of previously executed statements
  A nice feature here is the “describe under cursor�? which shows the table structure you a currently querying.
• Schema Browser to show tables, view, indexes , sequences, synonyms, pl/sql and triggers for a particular schema.

Here is a screenshot showcasing some of these features. TOra supports Database versions up to Oracle 9i (which release is not specified). Being connected to an Oracle 10g database didn’t seem to cause any problems. I installed a trial version on a Windows platform and played around with that for a while. The first thing that strikes me is the resemblance to Toad. There are a lot of similarities between these two products. The overall look and feel, where the different tools are located etc. make clear that TOra was inspired by Toad. My experience with TOra… it has a lot of features I never use. The ones I do use, don’t provide me with the feedback I need. An example to illustrate this: If I create a procedure with an error in it. It will compile, or at least it appears that way. The error messages are shown on the status bar and disappear after a while. You can recall the messages using a button on the status bar, or navigate the cursor to the status bar to display the error message in a tooltip. What I’d like to see is more immediate feedback to notice errors early on during development. Toad will display a pop-up window clearly stating the error. Creating and manipulating Objects formed somewhat of a problem in the SQL Worksheet. A valid Object Type Body definition (tested in SQL*Plus) resulted in an “ORA-00900: Invalid SQL Statement” error, making it impossible to create the Object Type Body here. Doing a similar action(creating a Object Type Body in a SQL window) in Toad or SQL*Plus was no problem. A valid Object Type Body was the result. A really nice feature in TOra is the DB Extract/Compare/Search tool. Simply using check-marks to specify which database objects you want to use and this tool will either Extract (creating installation scripts), Compare (handy if you need to compare two schema’s) or Search the database. I think it’s possible to overcome the limitations I mentioned before, once you get more comfortable using this tool. Getting used to a tool like Toad or TOra requires some time. There are so many tools at your disposal, learning each one of them simply takes time and effort. It’s like a new pair of shoes, once you break them in, they’re comfortable to wear, but the first two weeks… There are a number of tools on the market to choose from, especially if you’re using Windows. TOra beats Toad price-wise, but for how long? Quest is involved in TOra, draw your own conclusion. How will it compete with others on the Windows platform? Is it still going to evolve and incorporate new features and enhancements? If you’re not on a Windows platform, TOramight be worth looking into. The price is right, it offers a lot (maybe most) of the features Toad has. Jealousy can be a thing of the past.

Although this approach is often appropriate, there are alternatives. In this recipe, you will learn alternative techniques that rely on variable binding and Oracle’s CURSOR expressions and measure these against the traditional approach. All the code is included, so you should be able to replicate the testing conditions and do some benchmarking of your own.

Setting Up the Data Structure

For this recipe I am going to reuse the example from the “Inserting Multiple Rows With One Database Call Using PHP and Oracle Collections” recipe that illustrated how to use Oracle cursors to speed up bulk data inserts. In that recipe, you created a simple database structure that could be used to support a hypothetical e-commerce Web application. Here you will add another table that can be used to track status changes to each order (Order received -> Order processed -> Order packed -> Order shipped). In other words, by looking at this table, you will be able to tell the current status, as well as the history of status changes, for any given order.

Here is the rehashed data structure:

Here is the SQL code to create the tables above:

CREATE TABLE CUSTOMERS (
  CUSTOMER_ID    NUMBER        NOT NULL,
  CUSTOMER_NAME  VARCHAR2 (100)  NOT NULL,
  CONSTRAINT PK_CUSTOMERS
  PRIMARY KEY ( CUSTOMER_ID ) ) ;
/
CREATE TABLE ITEMS (
  ITEM_ID           NUMBER        NOT NULL,
  ITEM_DESCRIPTION  VARCHAR2 (50)  NOT NULL,
  ITEM_PRICE        NUMBER        NOT NULL,
  CONSTRAINT PK_ITEMS
  PRIMARY KEY ( ITEM_ID ) ) ;
/
CREATE TABLE ORDERS (
  ORDER_ID     NUMBER        NOT NULL,
  ORDER_DATE   DATE          NOT NULL,
  CUSTOMER_ID  NUMBER,
  CONSTRAINT PK_ORDERS
  PRIMARY KEY ( ORDER_ID ) ) ; 

ALTER TABLE ORDERS ADD  CONSTRAINT FK_CUSTOMER
 FOREIGN KEY (CUSTOMER_ID)
  REFERENCES CUSTOMERS (CUSTOMER_ID) ;
/
CREATE TABLE ORDER_ITEMS (
  ORDER_LINE_ID  NUMBER        NOT NULL,
  ORDER_ID       NUMBER        NOT NULL,
  ITEM_ID        NUMBER        NOT NULL,
  CONSTRAINT PK_ORDER_ITEMS
  PRIMARY KEY ( ORDER_LINE_ID ) ) ; 

ALTER TABLE ORDER_ITEMS ADD  CONSTRAINT FK_ITEM
 FOREIGN KEY (ITEM_ID)
  REFERENCES ITEMS (ITEM_ID) ;

ALTER TABLE ORDER_ITEMS ADD  CONSTRAINT FK_ORDER
 FOREIGN KEY (ORDER_ID)
  REFERENCES ORDERS (ORDER_ID) ;

/
CREATE TABLE ORDER_TRACKING (
  RECORD_ID       NUMBER        NOT NULL,
  CURRENT_STATUS  VARCHAR2 (100),
  ORDER_ID        NUMBER,
  RECORD_DATE     DATE,
  CONSTRAINT PK_ORDER_TRACKING
  PRIMARY KEY ( RECORD_ID ) ) ; 

ALTER TABLE ORDER_TRACKING ADD  CONSTRAINT FK_ORDER_TRACKING
 FOREIGN KEY (ORDER_ID)
  REFERENCES ORDERS (ORDER_ID) ;

CREATE SEQUENCE ORDER_LINE_SQ
INCREMENT BY 1;
/

CREATE SEQUENCE ORDER_STATUS_SQ
INCREMENT BY 1;
/

And a sequence to make it simple to come up with a unique primary key whenever a row is inserted in the Order_Items table:

CREATE SEQUENCE ORDER_LINE_SQ
INCREMENT BY 1;
/

And here’s the code to populate the test data. Assume that the CUSTOMERS table is pre-populated with the customer CUSTOMER_ID = 1 and the ITEMS table contains 15 items with ITEM_IDs from 1 to 15:

//connect to the database using your login name and password
$schema = "my_schema";
$ password = "my_password";

$db = oci_connect('$schema, $password) or die;

$arr_order_items = array(1,2,3,4,5,6,7,8,9,10,11,12,13,14,15); //all items on the list
$arr_status = array("Received","Processed","Packed","Shipped");

//create 200 orders for CUSTOMER_ID = 1

for ($order_id = 1;$order_id<=200;$order_id++) {
   $sql = "INSERT INTO ORDERS (order_id,customer_id,order_date) VALUES (".$order_id.",1,SYSDATE)";
   $stmt = oci_parse($db,$sql);
   oci_execute($stmt);
//add 15 items to each order
   foreach ($arr_order_items as $item) {
      $sql = "INSERT INTO ORDER_ITEMS (order_line_id,order_id,item_id) VALUES
      (ORDER_LINE_SQ.nextval,".$order_id.",".$item.")";
      $stmt = oci_parse($db,$sql);
      oci_execute($stmt);
   }
//populate status for each order
   foreach ($arr_status as $status) {
      $sql = "INSERT INTO ORDER_TRACKING (order_id,current_status,record_date) VALUES
      (".$order_id.",'".$status."',SYSDATE)";
      $stmt = oci_parse($db,$sql);
      oci_execute($stmt);
  }
}

At this point the ORDERS table should be populated with 200 orders placed by customer 1. Each order comes with 15 items in the ORDER_ITEMS table and 4 statuses in the ORDER_TRACKING table.

Report Requirements

First, create an order history report for your only customer (CUSTOMER_ID = 1). The report will present the complete list of customer’s orders with two subsections: one that shows the items included with each given order, and one that lists this order’s status update history. In the following sections, you will create this report using three different techniques: by executing a series of stand-alone database calls in loop, by executing the same stand-alone queries with variable binding, and by using Oracle’s CURSOR expressions in a single query.

Simple sub-queries. This approach is, no doubt, the most traditional way of building complex reports. In this example you will execute a simple query to retrieve the list orders for given customer id and then loop through the list of orders and execute two sub-queries, one to get the status change history and the other to get the item list. The results will be stored in an associative array $arr_orders and later presented as an HTML table.

$stmt = oci_parse($db,"SELECT order_id,order_date,customer_name FROM CUSTOMERS,ORDERS WHERE
ORDERS.customer_id = 1 AND ORDERS.CUSTOMER_ID = CUSTOMERS.CUSTOMER_ID");

oci_execute($stmt);

while (oci_fetch_assoc($stmt,$orders)) {
   $arr_order_status = array(); // to temporarily store status history for given order id
   $arr_order_items = array(); // to temporarily store item list for given order id
   $arr_order_line = array(); // to temporarily store complete order information for given order id

$arr_order_line =    array("ORDER_ID"=>$orders["ORDER_ID"],"ORDER_DATE"=>$orders["ORDER_DATE"],
   "CUSTOMER_NAME"=>$orders["CUSTOMER_NAME"]);

//first "sub-query"
   $stmt2 = oci_parse($db,"SELECT current_status,record_date FROM ORDER_TRACKING WHERE order_id = ".$orders["ORDER_ID"]);

oci_execute($stmt2);

while (oci_fetch_assoc($stmt2,$status)) { //note that this function is PHP5 only, use OCIFetchInto for PHP4
      $arr_order_status[] = array("CURRENT_STATUS"=>$status["CURRENT_STATUS"],
      "RECORD_DATE"=>$status[ "RECORD_DATE" ]);
   }

$arr_order_line["STATUS"] = $arr_order_status;

//second "sub-query"
   $stmt3 = Oci_parse($db,"SELECT item_description,item_price FROM ITEMS,ORDER_ITEMS WHERE
   ORDER_ITEMS.ITEM_ID = ITEMS.ITEM_ID AND ORDER_ITEMS.ORDER_ID = ".$orders["ORDER_ID"]);
   oci_execute($stmt3);
   while (oci_fetch_assoc($stmt3,$items)) {
      $arr_order_items[] = array("ITEM_DESCRIPTION"=>$items["ITEM_DESCRIPTION"],
      "ITEM_PRICE"=>$items["ITEM_PRICE"]);
   }

$arr_order_line["ITEMS"] = $arr_order_items;

//add new line to the orders array
   $arr_orders[] = $arr_order_line;
}

Simple sub-queries with variable binding. This approach is similar to the previous technique; it also relies on executing stand-alone database queries in a loop. However, this time you will use variable binding. In theory, using variable binding should result in faster query execution, since Oracle wouldn’t have to parse each and every new statement. Also, binding is generally safer, since it reduces the risk of a SQL injection attack.

$stmt = oci_parse($db,"SELECT order_id,order_date,customer_name FROM CUSTOMERS,
                  ORDERS WHERE ORDERS.customer_id = 1 AND ORDERS.CUSTOMER_ID = CUSTOMERS.CUSTOMER_ID");

//parse the statement to get status update history and bind it to the variable
$stmt2 = oci_parse($db,"SELECT current_status,record_date FROM ORDER_TRACKING WHERE order_id = :var");
oci_bind_by_name($stmt2, ":var", $order_id,32);

//parse the statement to get item list and bind it to the variable
$stmt3 = oci_parse($db,"SELECT item_description,item_price FROM ITEMS,
                   ORDER_ITEMS WHERE ORDER_ITEMS.ITEM_ID = ITEMS.ITEM_ID AND ORDER_ITEMS.ORDER_ID = :var");
oci_bind_by_name ($stmt3, ":var", $order_id,32);

oci_execute($stmt); //execute order list statement

while (OCIFetchInto($stmt,$orders)) {
   $arr_order_status = array();
   $arr_order_items = array();
   $arr_order_line = array();

$arr_order_line = array("ORDER_ID"=>$orders["ORDER_ID"],"ORDER_DATE"=>$orders["ORDER_DATE"],
   "CUSTOMER_NAME"=>$orders["CUSTOMER_NAME"]);

$order_id = $orders["ORDER_ID"]; //current order id

//first "sub-query"
   oci_execute($stmt2);
   while (oci_fetch_assoc($stmt2,$status)) {
      $arr_order_status[] = array("CURRENT_STATUS"=>$status["CURRENT_STATUS"],
      "RECORD_DATE"=>$status[ "RECORD_DATE" ]);
   }

$arr_order_line["STATUS"] = $arr_order_status;

//second "sub-query"
   oci_execute($stmt3);
   while (oci_fetch_assoc($stmt3,$items)) {
      $arr_order_items[] = array("ITEM_DESCRIPTION"=>$items["ITEM_DESCRIPTION"],
      "ITEM_PRICE"=>$items["ITEM_PRICE"]);
   }

$arr_order_line["ITEMS"] = $arr_order_items;

//add new line to the orders array
   $arr_orders[] = $arr_order_line;
}

Single query with Oracle CURSOR sub-queries. The third approach involves using Oracle’s CURSOR expressions in a single database call. Even though there is still quite a bit of looping involved, this time, instead of querying the original database tables you will query the collections included in the initial call.

$stmt = oci_parse($db,"SELECT order_id,order_date,customer_name,
CURSOR(SELECT current_status,record_date FROM ORDER_TRACKING WHERE order_id = ORDERS.order_id) as status,
CURSOR(SELECT item_description,item_price FROM ITEMS,
       ORDER_ITEMS WHERE ORDER_ITEMS.ITEM_ID = ITEMS.ITEM_ID AND ORDER_ITEMS.ORDER_ID = ORDERS.order_id) AS items
FROM CUSTOMERS, ORDERS WHERE ORDERS.customer_id = 1
AND ORDERS.CUSTOMER_ID = CUSTOMERS.CUSTOMER_ID");

oci_execute($stmt);
while (oci_fetch_assoc ($stmt,$orders)) {
   $arr_order_status = array();
   $arr_order_items = array();
   $arr_order_line = array();

$arr_order_line = array("ORDER_ID"=>$orders["ORDER_ID"],"ORDER_DATE"=>$orders["ORDER_DATE"],
   "CUSTOMER_NAME"=>$orders["CUSTOMER_NAME"]);

//process cursors
   oci_execute($orders["STATUS"]); //get data from the nested collection
   while (oci_fetch_assoc($orders["STATUS"],$status)) {
      $arr_order_status[] = array("CURRENT_STATUS"=>$status["CURRENT_STATUS"],
      "RECORD_DATE"=>$status["RECORD_DATE"]);
   }

$arr_order_line["STATUS"] = $arr_order_status;

oci_execute($orders["ITEMS"]);
   while (OCIFetchinto($orders[ "ITEMS" ],$items,OCI_ASSOC)) {
      $arr_order_items[] = array("ITEM_DESCRIPTION"=>$items["ITEM_DESCRIPTION"],
      "ITEM_PRICE"=>$items["ITEM_PRICE"]);
   }

$arr_order_line["ITEMS"] = $arr_order_items;

//add new line to the orders array
   $arr_orders[] = $arr_order_line;
}

Viewing the Report

The report data is now contained in the $arr_orders array. Here is a little snippet of code that loops through the array and renders the data as an HTML table for easy viewing. Since in our example all orders are identical, this snippet will only show the first two orders.

for($i=0;$i<=1;$i++) { //show only the first two orders
   echo "<table cellpadding=\"2\" cellspacing=\"2\" width=\"500\" border=\"1\">";
   echo "<tr><td colspan=\"2\">Order # ".($i+1)." (".$arr_orders[$i]["ORDER_ID"]." "
   .$arr_orders[$i]["ORDER_DATE"].",".$arr_orders[$i]["CUSTOMER_NAME"].")</td></tr>";
   echo "<tr><td valign=\"top\">";
   foreach($arr_orders[$i]["STATUS"] as $status) {
      echo $status["RECORD_DATE"]." - ".$status["CURRENT_STATUS"]."<br />";
   }
   echo "</td><td valign=\"top\">";
   foreach($arr_orders[$i]["ITEMS"] as $item) {
      echo $item["ITEM_DESCRIPTION"]." - ".$item["ITEM_PRICE"]."<br />";
   }
   echo "</td></tr></table>";
}

Benchmarking

Finally, review the results and decide which method would be the most efficient way to produce reports for your system. Keep in mind, however, that no two systems are identical and there are many factors to consider while optimizing for best performance. Some factors may be database related (like fine-tuning the indexes), some may involve the PHP or network optimization, and so on. As to the simple system discussed here, feel free to draw your own conclusions by looking at the tables below:

200 orders

1,000 orders

Conclusion

There are, of course, more ways to provide same functionality–for example, by using an Oracle stored procedure with REF CURSORs to output the data. As you can see, in this particular test, it turned out that using stand-alone sub-queries would provide better performance. However, this test is by no means comprehensive and I encourage you to use the code presented in this recipe to create your own tests and find out which method works best for you.

Oracle is the most popular commercial database used with PHP. There are many ways of accessing Oracle databases in PHP. These include:

• The oracle extension
• The oci8 extension
• PEAR DB library
• ADOdb library

The wide range of choices is confusing to someone just starting with Oracle and PHP. I will briefly summarize the differences, and show you the advantages of using ADOdb.

First we have the C extensions which provide low-level access to Oracle functionality. These C extensions are precompiled into PHP, or linked in dynamically when the web server starts up. Just in case you need it, here’s a guide to installing Oracle and PHP on Linux.

Here is an example of using the oci8 extension to query the emp table of the scott schema with bind parameters:

$conn = OCILogon("scott","tiger", $tnsName); 

$stmt = OCIParse($conn,"select * from emp where empno > :emp order by empno");
$emp = 7900;
OCIBindByName($stmt, ':emp', $emp);
$ok = OCIExecute($stmt);
while (OCIFetchInto($stmt,$arr)) {
	print_r($arr);
	echo "<hr>";
}

This generates the following output:

Array ( [0] => 7902 [1] => FORD [2] => ANALYST [3] => 7566 [4] => 03/DEC/81 [5] => 3000 [7] => 20 )

We also have many higher level PHP libraries that allow you to simplify the above code. The most popular are PEAR DB and ADOdb. Here are some of the differences between these libraries:

PEAR DB is good enough for simple web apps. But if you need more power, you can see ADOdb offers more sophisticated functionality. The rest of this article will concentrate on using ADOdb with Oracle. You can find out more about connecting to Oracle later in this guide.

ADOdb Example

In ADOdb, the above oci8 example querying the emp table could be written as:

include "/path/to/adodb.inc.php";
$db = NewADOConnection("oci8");
$db->Connect($tnsName, "scott", "tiger");

$rs = $db->Execute("select * from emp where empno>:emp order by empno",
                    array('emp' => 7900));
while ($arr = $rs->FetchRow()) {
    print_r($arr);
	echo "<hr>";
}

The Execute( ) function returns a recordset object, and you can retrieve the rows returned using $recordset->FetchRow( ).

If we ignore the initial connection preamble, we can see the ADOdb version is much easier and simpler:

$stmt = OCIParse($conn,
       "select * from emp where empno > :emp");
$emp = 7900;
OCIBindByName($stmt, ':emp', $emp);
$ok = OCIExecute($stmt);

while (OCIFetchInto($stmt,$arr)) {
	print_r($arr);
	echo "<hr>";
} 

$recordset = $db->Execute("select * from emp where empno>:emp",
                           array('emp' => 7900));

while ($arr = $recordset->FetchRow()) {
	print_r($arr);
	echo "<hr>";
}

2. ADOdb Query Semantics

You can also query the database using the standard Microsoft ADO MoveNext( ) metaphor. The data array for the current row is stored in the fields property of the recordset object, $rs. MoveNext( ) offers the highest performance among all the techniques for iterating through a recordset:

$rs = $db->Execute("select * from emp where empno>:emp", array('emp' => 7900));
while (!$rs->EOF) {
	print_r($rs->fields);
	$rs->MoveNext();
}

And if you are interested in having the data returned in a 2-dimensional array, you can use:

$arr = $db->GetArray("select * from emp where empno>:emp", array('emp' => 7900));

Now to obtain only the first row as an array:

$arr = $db->GetRow("select * from emp where empno=:emp", array('emp' => 7900));

Or to retrieve only the first field of the first row:

$arr = $db->GetOne("select ename from emp where empno=:emp", array('emp' => 7900));

For easy pagination support, we provide the SelectLimit function. The following will perform a select query, limiting it to 100 rows, starting from row 201 (row 1 being the 1st row):

$offset = 200; $limitrows = 100;
$rs = $db->SelectLimit('select * from table', $limitrows, $offset);

The $offset parameter is optional.

Array Fetch Mode

When data is being returned in an array, you can choose the type of array the data is returned in.

1. Numeric indexes – use $connection->SetFetchMode(ADODB_FETCH_NUM).
2. Associative indexes – the keys of the array are the names of the fields (in upper-case). Use $connection->SetFetchMode(ADODB_FETCH_ASSOC).
3. Both numeric and associative indexes – use $connection->SetFetchMode(ADODB_FETCH_BOTH).

The default is ADODB_FETCH_BOTH for Oracle.

Caching

You can define a database cache directory using $ADODB_CACHE_DIR, and cache the results of frequently used queries that rarely change. This is particularly useful for SQL with complex where clauses and group-by’s and order-by’s. It is also good for relieving heavily-loaded database servers.

This example will cache the following select statement for 3600 seconds (1 hour):

$ADODB_CACHE_DIR = '/var/adodb/tmp';
$rs = $db->CacheExecute(3600, "select names from allcountries order by 1");

There are analogous CacheGetArray( ), CacheGetRow( ), CacheGetOne( ) and CacheSelectLimit( ) functions. The first parameter is the number of seconds to cache. You can also pass a bind array as a 3rd parameter (not shown above).There is an alternative syntax for the caching functions. The first parameter is omitted, and you set the cacheSecs property of the connection object:

$ADODB_CACHE_DIR = '/var/adodb/tmp';
$connection->cacheSecs = 3600;
$rs = $connection->CacheExecute($sql, array('id' => 1));

3. Using Prepare( ) For Frequently Used Statements

Prepare( ) is for compiling frequently used SQL statement for reuse. For example, suppose we have a large array which needs to be inserted into an Oracle database. The following will result in a massive speedup in query execution (at least 20-40%), as the SQL statement only needs to be compiled once:

$stmt = $db->Prepare('insert into table (field1, field2) values (:f1, :f2)');
foreach ($arrayToInsert as $key => $value) {
	$db->Execute($stmt, array('f1' => $key, 'f2' => $val);
}

4. Working With LOBs

Oracle treats data which is more than 4000 bytes in length specially. These are called Large Objects, or LOBs for short. Binary LOBs are BLOBs, and character LOBs are CLOBs. In most Oracle libraries, you need to do a lot of work to process LOBs, probably because Oracle designed it to work in systems with little memory. ADOdb tries to make things easy by assuming the LOB can fit into main memory.

ADOdb will transparently handle LOBs in select statements. The LOBs are automatically converted to PHP variables without any special coding.

For updating records with LOBs, the functions UpdateBlob( ) and UpdateClob( ) are provided. Here’s a BLOB example. The parameters should be self-explanatory:

$ok = $db->Execute("insert into aTable (id, name, ablob)
                                values (aSequence.nextVal, 'Name', null)");
if (!$ok) return LogError($db->ErrorMsg());
# params: $tableName, $blobFieldName, $blobValue, $whereClause
$db->UpdateBlob('aTable', 'ablob', $blobValue, 'id=aSequence.currVal');

and the analogous CLOB example:

$ok = $db->Execute("insert into aTable (id, name, aclob)
                                values (aSequence.nextVal, 'Name', null)");
if (!$ok) return LogError($db->ErrorMsg());
$db->UpdateClob('aTable', 'aclob', $clobValue, 'id=aSequence.currVal');

Note that LogError( ) is a user-defined function, and not part of ADOdb.

Inserting LOBs is more complicated. Since ADOdb 4.55, we allow you to do this (assuming that the photo field is a BLOB, and we want to store $blob_data into this field, and the primary key is the id field):

$sql = "INSERT INTO photos ( ID, photo) ".
			"VALUES ( :id, empty_blob() )".
			" RETURNING photo INTO  x";

$stmt = $db->PrepareSP($sql);
		$db->InParameter($stmt, $id, 'id');
		$blob = $db->InParameter($stmt, $blob_data, 'xx',-1, OCI_B_BLOB);
		$db->StartTrans();
		$ok = $db->Execute($stmt);
		$db->CompleteTrans();

5. REF CURSORs

Oracle recordsets can be passed around as variables called REF Cursors. For example, in PL/SQL, we could define a function open_tab that returns a REF CURSOR in the first parameter:

TYPE TabType IS REF CURSOR RETURN TAB%ROWTYPE;

PROCEDURE open_tab (tabcursor IN OUT TabType,tablenames IN VARCHAR) IS
	BEGIN
		OPEN tabcursor FOR SELECT * FROM TAB WHERE tname LIKE tablenames;
	END open_tab;

In ADOdb, we could access this REF Cursor using the ExecuteCursor() function. The following will find all table names that begin with ‘A’ in the current schema:

$rs = $db->ExecuteCursor("BEGIN open_tab(:refc,'A%'); END;",'refc');
while ($arr = $rs->FetchRow()) print_r($arr);

The first parameter is the PL/SQL statement, and the second parameter is the name of the REF Cursor.

6. In and Out Parameters

The following PL/SQL stored procedure requires an input variable, and returns a result into an output variable:

PROCEDURE data_out(input IN VARCHAR, output OUT VARCHAR) IS
	BEGIN
		output := 'I love '||input;
	END;

The following ADOdb code allows you to call the stored procedure:

$stmt = $db->PrepareSP("BEGIN adodb.data_out(:a1, :a2); END;");
$input = 'Sophia Loren';
$db->InParameter($stmt,$input,'a1');
$db->OutParameter($stmt,$output,'a2');
$ok = $db->Execute($stmt);
if ($ok) echo ($output == 'I love Sophia Loren') ? 'OK' : 'Failed';

PrepareSP( ) is a special function that knows about bind parameters. The main limitation currently is that IN OUT parameters do not work.

Bind Parameters and REF CURSORs

We could also rewrite the REF CURSOR example to use InParameter (requires ADOdb 4.53 or later):

$stmt = $db->PrepareSP("BEGIN adodb.open_tab(:refc,:tabname); END;");
$input = 'A%';
$db->InParameter($stmt,$input,'tabname');
$rs = $db->ExecuteCursor($stmt,'refc');
while ($arr = $rs->FetchRow()) print_r($arr);

Bind Parameters and LOBs

You can also operate on LOBs. In this example, we have IN and OUT parameters using CLOBs.

$text = 'test test test';
	$sql = "declare rs clob; begin :rs := lobinout(:sa0); end;";
	$stmt = $conn -> PrepareSP($sql);
	$conn -> InParameter($stmt,$text,'sa0', -1, OCI_B_CLOB); # -1 means variable length
	$rs = '';
	$conn -> OutParameter($stmt,$rs,'rs', -1, OCI_B_CLOB);
	$conn -> Execute($stmt);
	echo "return = ".$rs."<br>";

Similarly, you can use the constant OCI_B_BLOB to indicate that you are using BLOBs.

Reusing Bind Parameters with CURSOR_SHARING=FORCE

Many web programmers do not care to use bind parameters, and prefer to enter the SQL directly. So instead of:

$arr = $db->GetArray("select * from emp where empno>:emp", array('emp' => 7900));

They prefer entering the values inside the SQL:

$arr = $db->GetArray("select * from emp where empno>7900");

This reduces Oracle performance because Oracle will reuse compiled SQL which is identical to previously compiled SQL. The above example with the values inside the SQL is unlikely to be reused. As an optimization, from Oracle 8.1 onwards, you can set the following session parameter after you login:

ALTER SESSION SET CURSOR_SHARING=FORCE

This will force Oracle to convert all such variables (eg. the 7900 value) into constant bind parameters, improving SQL reuse.

More speedup tips.

7. Dates and Datetime in ADOdb

There are two things you need to know about dates in ADOdb.

First, to ensure cross-database compability, ADOdb assumes that dates are returned in ISO format (YYYY-MM-DD H24:MI:SS).

Secondly, since Oracle treats dates and datetime as the same data type, we decided not to display the time in the default date format. So on login, ADOdb will set the NLS_DATE_FORMAT to ‘YYYY-MM-DD’. If you prefer to show the date and time by default, do this:

$db = NewADOConnection('oci8');
$db->NLS_DATE_FORMAT =  'RRRR-MM-DD HH24:MI:SS';
$db->Connect($tns, $user, $pwd);

Or execute:

$sql = "ALTER SESSION SET NLS_DATE_FORMAT = 'RRRR-MM-DD HH24:MI:SS'";
$db->Execute($sql);

If you are not concerned about date portability and do not use ADOdb’s portability layer, you can use your preferred date format instead.

8. Database Portability Layer

ADOdb provides the following functions for portably generating SQL functions as strings to be merged into your SQL statements:

ADOdb also provides multiple oracle oci8 drivers for different scenarios:

Here’s an example of calling the oci8po driver. Note that the bind variables use question-mark:

$db = NewADOConnection('oci8po');
$db->Connect($tns, $user, $pwd);
$db->Execute("insert into atable (f1, f2) values (?,?)", array(12, 'abc'));

9. Connecting to Oracle

Before you can use ADOdb, you need to have the Oracle client installed and setup the oci8 extension. This extension comes pre-compiled for Windows (but you still need to enable it in the php.ini file). For information on compiling the oci8 extension for PHP and Apache on Unix, there is an excellent guide at oracle.com.

Should You Use Persistent Connections

One question that is frequently asked is should you use persistent connections to Oracle. Persistent connections allow PHP to recycle existing connections, reusing them after the previous web pages have completed. Non-persistent connections close automatically after the web page has completed. Persistent connections are faster because the cost of reconnecting is expensive, but there is additional resource overhead. As an alternative, Oracle allows you to pool and reuse server processes; this is called Shared Server (also known as MTS).

The author’s benchmarks suggest that using non-persistent connections and the Shared Server configuration offer the best performance. If Shared Server is not an option, only then consider using persistent connections.

Connection Examples

Just in case you are having problems connecting to Oracle, here are some examples:

a. PHP and Oracle reside on the same machine, use default SID, with non-persistent connections:

$conn = NewADOConnection('oci8');
	$conn->Connect(false, 'scott', 'tiger');

b. TNS Name defined in tnsnames.ora (or ONAMES or HOSTNAMES), eg. ‘myTNS’, using persistent connections:

$conn = NewADOConnection('oci8');
	$conn->PConnect(false, 'scott', 'tiger', 'myTNS');

or

$conn->PConnect('myTNS', 'scott', 'tiger');

c. Host Address and SID

$conn->connectSID = true;
	$conn->Connect('192.168.0.1', 'scott', 'tiger', 'SID');

d. Host Address and Service Name

$conn->Connect('192.168.0.1', 'scott', 'tiger', 'servicename');

e. Oracle connection string:

$cstr = "(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=$host)(PORT=$port))
			(CONNECT_DATA=(SID=$sid)))";
	$conn->Connect($cstr, 'scott', 'tiger');

f. ADOdb data source names (dsn):

$dsn = 'oci8://user:pwd@tnsname/?persist';  # persist is optional
	$conn = ADONewConnection($dsn);  # no need for Connect/PConnect

$dsn = 'oci8://user:pwd@host/sid';
	$conn = ADONewConnection($dsn);

$dsn = 'oci8://user:pwd@/';   # oracle on local machine
	$conn = ADONewConnection($dsn);

With ADOdb data source names, you don’t have to call Connect( ) or PConnect( ).

10. Error Checking

The examples in this article are easy to read but a bit simplistic because we ignore error-handling. Execute( ) and Connect( ) will return false on error. So a more realistic way to call Connect( ) and Execute( ) is:

function InvokeErrorHandler()
{
global $db; ## assume global
    MyLogFunction($db->ErrorNo(), $db->ErrorMsg());
}
if (!$db->Connect($tns, $usr, $pwd)) InvokeErrorHandler();

$rs = $db->Execute("select * from emp where empno>:emp order by empno",
                    array('emp' => 7900));
if (!$rs) return InvokeErrorHandler();
while ($arr = $rs->FetchRow()) {
    print_r($arr);
	echo "<hr>";
}

You can retrieve the error message and error number of the last SQL statement executed from ErrorMsg( ) and ErrorNo( ). You can also define a custom error handler function. ADOdb also supports throwing exceptions in PHP5.

Handling Large Recordsets (added 27 May 2005)

The oci8 driver does not support counting the number of records returned in a SELECT statement, so the function RecordCount() is emulated when the global variable $ADODB_COUNTRECS is set to true, which is the default. We emulate this by buffering all the records. This can take up large amounts of memory for big recordsets. Set $ADODB_COUNTRECS to false for the best performance.This variable is checked every time a query is executed, so you can selectively choose which recordsets to count.

11. Other ADOdb Features

Schema generation. This allows you to define a schema using XML and import it into different RDBMS systems portably.

Performance monitoring and tracing. Highlights of performance monitoring include identification of poor and suspicious SQL, with explain plan support, and identifying which web pages the SQL ran on.

12. Download

You can download ADOdb from sourceforge. ADOdb uses a BSD style license. That means that it is free for commercial use, and redistribution without source code is allowed.

13. Resources

• Oracle’s Hitchhiker Guide to PHP
• OTN article on Optimizing PHP and Oracle by this author.
• Oracle has an excellent FAQ on PHP
• PHP oci8 manual pages
• ADOdb forums.

Most web applications have connections to databases and use XML to transfer data from the database to the web application and vice versa. Every major database vendor has proprietary extensions for using XML with relational databases, but they take completely different approaches, and there is no interoperability between them. Developers need to be able to write applications that work for databases from multiple vendors.

XQuery and SQL/XML are two standards that use declarative, portable queries to return XML by querying data. In both standards, the XML can have any desired structure, and the queries can be arbitrarily complex. XQuery is XML-centric, while SQL/XML is SQL-centric.

SQL/XML is an extension of SQL that is part of ANSI/ISO SQL 2003. It lets SQL queries create XML structures with a few powerful XML publishing functions. For a SQL programmer, SQL/XML easy to learn because it involves only a few small additions to the existing SQL language. Since SQL is a mature language, there are a lot of tools and infrastructure for SQL. For instance, SQL/XML uses JDBC to return results, and there will soon be a standard API for XQuery. SQL also has functionality soon to be found in XQuery, such as updates and stored procedures.

Note: SQL/XML is completely different from Microsoft’s SQLXML, a proprietary technology used in SQL Server. The similarity in names has caused a great deal of confusion in the industry.

XQuery is a completely new query language that uses XML as the basis for its data model and type system. It is being developed in the XML Query Working Group, which is a part of the World Wide Web Consortium. In this paper, we characterize XQuery as a “Native XML Programming Language”. XQuery is based on XML in the same way that SQL is based on the relational model or object oriented languages are based on the object oriented model – XML is central to its type system, in which elements and attributes are just as fundamental as integers and strings. Although XQuery per se has no concept of relational data, several products and many projects provide ways to query relational data using an XML view of the database, and the need to make this possible has influenced the design of XQuery throughout its development. XQuery allows you to work in the XML world no matter what type of data you’re working with – relational, XML or object data. XQuery is ideal for native XML programming. When used with XML views of relational data, it is also ideal for queries data that must represent results as XML, to query XML stored inside or outside the database, or to span relational and XML sources.

For queries based only on relational data, SQL/XML and XQuery have substantially similar functionality. However, the way in which a given task is done is quite different, since SQL/XML operates on the borderline between SQL and XML, and XQuery lives in a purely XML world. Even when the data is all relational, the two languages appeal to very different audiences – SQL/XML is very much an extension of SQL, designed for SQL programmers, and XQuery takes a purely XML view of the world. For queries that span relational and XML sources, XQuery has important advantages.

This talk uses a series of concrete queries written in each language to show the advantages of each. It explains why we need both languages, discussing the ways in which the languages differ and in which they overlap. It also explores the role of SQL/XML mappings as a way of creating XML views for XQuery.

XML and Relational – Opposites Attract

XML and relational databases are tightly wed in most web applications, but a look at the two models shows that it is an unlikely marriage – though a necessary one. The relational model is based on two dimensional tables which have neither hierarchy nor significant order. XML is based on trees in which order is significant. In the relational model, neither hierarchy nor sequence may be used to model information; in XML, hierarchy and sequence are the main ways to represent information. Although this is one of the more fundamental differences between the two models, it is by no means the only one.

In many environments, the same information is represented in relational databases when it is stored or queried, but in XML when it is exchanged or displayed on web pages. These representations are often completely different due to the differences in the models.

On web pages, XML is useful because the structure of XML closely matches the structure used to display the same information in HTML. If you look at web pages, they often use a distinctly hierarchical structure to present data for users – after all, users don’t want to look at a bunch of tables and do joins in their head. But most of the data for these web pages comes from relational databases, and needs to be converted to appropriate XML hierarchies.

For web messages, the format of a web message is often specified by a standards organization or a trade partner, and these formats are generally hierarchical. Again, the data for a web message generally comes from relational data, and the consumer of a web message often needs to put data into a relational database.

For instance, suppose a consulting company needs to represent a set of projects and the companies for whom the projects are being done. In a relational database, this might be represented by the following tables:

In SQL, if we want to see the projects associated with each customer, we would do the following query:

Here is the output of the above SQL query:

Suppose we want to translate this information into XML in order to publish it to the Web, in a document, or in a web message. Like most XML applications, we will leverage the hierarchy of XML to express relationships, listing the projects for each customer within the element that represents the customer:

Note that in the original SQL tables, each customer is represented only once. This is also true of the XML. The SQL result set, however, contains multiple rows for a given customer if that customer is associated with more than one project, and these rows contain duplicate information. Translating this result set into the desired XML is tedious for the programmer. And just as a single relational database may be used with an infinite number of queries, it may also be used to create an infinite number of XML documents with different structures. Today, many programmers spend a great deal of time doing this kind of translation.

XML and Relational: Four Approaches

XML applications that use relational data can choose from four approaches, each with distinct advantages and disadvantages. The first three of these are compared in some detail, with code samples, in [SQL/XML-JDBC].

The programmer can use JDBC or ODBC together with SAX or DOM and perhaps XSLT to transform the results of SQL queries to XML. For instance, the program might first query for customers, then perform an additional query to find the projects associated with each customer. This is inefficient because of the number of queries required. Another approach would be to use SQL to create a table that lists customers and their projects, and pick through the rows to determine when a row represents a new customer. This requires more code, but is more efficient. Both of these approaches require significant amounts of tedious code, but they are often used when database independence is important.

The programmer can use the XML extensions provided by the major database vendors. These are based on several different approaches. Some of these are simpler to use or maintainable than others, but they all make the task easier. However, since these extensions are all proprietary, they are not an option when a database-independent solution is needed.

The programmer can use SQL/XML, which is part of SQL 2003. For a SQL programmer, this approach requires little new learning – a small set of XML publishing functions have been added to SQL to allow queries to create any desired XML structure. This approach will be explored with examples in the next section. SQL/XML is being supported by Oracle and IBM, but not by Microsoft. Database-independent implementations of SQL/XML are also available, and can be used with any major relational database. SQL/XML can be used with traditional database APIs such as JDBC.

The programmer can use XQuery, a native XML query language. Since XQuery is a new language, it requires more learning for SQL programmers, but it is likely to be more natural for XML programmers. Unlike SQL/XML, XQuery is optimal for processing XML, and it is also particularly good for applications that must process XML together with relational data, with full support for XML. Most of the major database vendors intend to support XQuery. The first standardized API for XQuery, XQuery for Java (JSR 225), is now being developed under Java Community Process, and is expected to be available shortly after the XQuery Recommendation is released.

SQL/XML

SQL/XML refers to the XML extensions of SQL. These are developed by INCITS H2.3, with participation from Oracle, IBM, Microsoft (which does not plan to implement SQL/XML), Sybase, and DataDirect Technologies. In SQL 2003, these extensions include:

• XML Publishing Functions
• The XML Datatype
• Mapping Rules

The XML Publishing Functions are the part that are directly used in a SQL query. The XML Datatype governs the result of a query, and the Mapping Rules determine how SQL data or metadata is represented as XML.

XML Publishing Functions

Let’s compare a traditional SQL query with one that uses an XML publishing function. Here is a traditional SQL query that shows customers and their associated projects:

Here is is an excerpt of the result:

Now let’s wrap the result in XML elements using xmlelement(), one of the publishing functions:

Each row in the result contains one Customer element. A Customer element looks like this:

xmlforest() is an XML publishing function that creates elements from a list of columns, using the name of the column as the name of the element. Using xmlforest() simplifies many queries significantly. For instance, the following query is equivalent to the previous one:

Now suppose we want to show customers and the projects associated with them. This is easily done with the following SQL query:

However, the result of this query is that shown in the CustomerProject table in the previous section, with one row for each Customer/Project pair. If a customer is associated with more than one project, there will be a row for that customer for each project. Here is a SQL/XML query that creates the XML equivalent to that table:

Here are the results of this query:

This is a straightforward XML translation of the that a SQL result set shown in the previous section, but for most XML applications it is not what we would want. Instead, we want to represent each customer once, with a list of that customer’s projects, as shown in the XML output in the previous section. In SQL/XML, this can be done by using a sub-query. Here is a subquery that retrieves the projects associated with each customer. In this subquery we use xmlattributes(), an XML publishing function that creates attributes within an element. The names of the attributes are taken from the names of the columns.

Here is the output of the above sub-query when c.CustId is 4:

This output contains two rows, with one element in each row. Subqueries in SQL/XML are allowed to return only one row; therefore, to return more than one row of values in a SQL/XML subquery, they must be combined to form a single value. xmlagg() is an XML publishing function that produces a forest of elements by collecting the XML values that are returned from multiple rows and concatenating the values to make one value. Here is a query that uses the above subquery to create the XML output from the previous section:

The above query illustrates a very common pattern used to create XML hierarchies using SQL/XML.

The XML Datatype

The XML Datatype is a datatype in the same way that integer, date, or CLOB are datatypes in SQL. Since SQL/XML allows a query to create XML instances, there must be a datatype that corresponds to these instances.

It is anticipated that the XML Datatype will be supported in JDBC 4.0. It is too early to say exactly how it will be used in that specification, but it is likely that it will retrieve XML values much like other values, and that XML values can be retrieved as text, DOM, or SAX events. This is the approach currently taken by DataDirect Connect for SQL/XML. To illustrate this, let’s use a SQL/XML query to create a table with two columns, an integer containing the CustId and an XML column containing the XML output from the previous query. Here is the query:

Suppose the above query is in a string called sqlxmlString. Then the following Java code can be used to execute the query and retrieve values.

The XML Type also plays a second important role – relational databases now routinely store XML in individual column, and the XML Type provides a standard type for such columns, which is useful both in SQL and in JDBC.

SQL/XML Mapping Rules

The XML publishing functions use SQL values to create XML values, and these XML values have W3C XML Schema types. When we discussed the XML publishing functions, we did not address specifically how the XML representation is determined. The mapping rules of SQL/XML describe in excruciating detail how SQL values can be mapped to and from XML values, and how SQL metadata can be mapped to and from W3C XML Schemas. To give a flavor for the level of detail in which this is specified, here are the equivalent headings from the SQL/XML specification’s table of contents:

• Mapping SQL character sets to Unicode.
• Mapping SQL <identifier>s to XML Names.
• Mapping SQL data types (as used in SQL-schemas to define SQL-schema objects such as columns) to XML Schema data types.
• Mapping values of SQL data types to values of XML Schema data types.
• Mapping an SQL table to an XML document and an XML Schema document.
• Mapping an SQL schema to an XML document and an XML Schema document.
• Mapping an SQL catalog to an XML document and an XML Schema document.
• Mapping Unicode to SQL character sets.
• Mapping XML Names to SQL <identifier>s.

These mappings can be parameterized in several ways, including the target namespace for the result, whether to handle nulls using xsi:nil or absence, and whether to map a table to a single element or a forest of elements. Here is an XML representation of the Customers table shown earlier, using a single element for each table and no target namespace:

Here is an XML representation of the same table using a forest of elements to represent each table:

These mappings are also defined on the metadata level. For instance, SQL/XML defines how the datatypes of SQL are represented in the equivalent XML Schema. Each SQL type is derived from an equivalent built-in W3C XML Schema type. Where needed, facets are used to represent constraints added to those of the base type:

As mentioned above, there are two ways to represent null values. Suppose the City column may have null values. Here is a row in the Customer’s table that represents a null value using the first strategy, a nilled element:

Here is a row that uses the second strategy, an absent element:

XQuery and Native XML Programming

The XQuery language was designed for querying or processing XML. Just as a traditional SQL query takes a set of tables as input and returns an XML table as its result, XQuery takes sequences of XML nodes as input and evaluates to a sequence of XML nodes. However, from the very beginning, XQuery was designed to allow XML views of non-XML data, as well as serialized forms of non-XML data. The reason for this is simple: XML is used to represent almost any conceivable kind of information, and it is easiest to integrate information if it is given a common view.

If everything looks like a nail, all you need is a hammer. Conventional Internet applications often store and query data using SQL, process data using Java or C#, and exchange data as XML. Using XQuery, it is possible to store, query, process, and exchange data as XML. This eliminates some of the mismatches that cause complications when working with XML in other environments.

Native XML Programming

XQuery is a language designed for integrating data from multiple sources, including XML sources like documents or web messages and databases. It does this by leveraging the ability of XML to model virtually any kind of data. To query anything with XQuery, it must be presented as though it were XML, either by serializing it as XML or by creating an XML view of the data through some form of middleware. For relational data, most systems use the SQL/XML mappings for the XML view, since they are quite suitable and have been specified in detail.

XML is the basis of XQuery’s type system and data model. The fundamental types of XQuery include the kinds of nodes found in XML documents: document nodes, elements, attributes, processing instructions, comments, and text nodes. XQuery also supports the built-in datatypes of W3C XML Schema for representing integers, strings, dates, and other datatypes – these built-in datatypes are predefined in XQuery, and are available with or without a schema.

Most modern programming languages provide some form of complex user-defined types, such as structures or objects. In XQuery, the only complex types are XML documents, elements, attributes, and W3C XML Schema complex types. There is no need to write a schema to create and manipulate complex XML structures in XQuery. However, if a query needs to ensure consistent use of the types in a schema, a schema may be imported into a query. This has an effect analogous to importing structure or class definitions in an object oriented language.

Programs tend to revolve around data, and the complex datatypes used in a language have a profound effect on the way that a language is used. As a result, languages are sometimes identified by the way they represent complex data; for instance, there are object-oriented languages and relational query languages. In this sense, XQuery can be considered Native XML Programming Language. XSLT and XPath are also Native XML Programming Languages. Most other languages used to process XML, including Java, C#, Perl, and Python are not. SQL/XML is fundamentally an extension to a relational query language, providing a bridge to XML.

The concept of a Native XML Programming Language is new, and many XML programmers are used to thinking of XML in terms of the constructs used in the languages with which they process XML. On XML-related mailing lists it is reasonably common to see beginners assert that XML is fundamentally relational or object-oriented, and even sophisticated XML programmers have been known to assert that XML is just text. In fact, the phrase “XML is Unicode with pointy brackets” has come to identify a vocal part of the XML community.

XML is not Objects!

An XML document can be represented using objects, and this is precisely the approach taken by DOM and JDOM. An XML parser can be used to create an appropriate object representation of an XML document without involving the programmer. However, the fundamental types of XML are not fundamental in object oriented languages, so casting and conversion is frequently required. Similarly, the basic notions of hierarchy and containment are not directly supported in the object oriented model, so explicit navigation is often required. This causes significant work for the programmer.

Adam Bosworth pointed this out with the following example. Suppose a programmer wants to compute price/earnings ratios from an XML feed. An individual stock might be represented as follows:

To compute the price/earnings ratio, we use the formula “pe = price / (revenues – expenses)”. To do this with the DOM, we also need to parse the XML, navigate to the places where this information is found, and convert the text of the document to the appropriate datatype. Here is the DOM code Adam provides for this:

This solution would have been much messier if Adam had not used the path expressions of XPath, a simple Native XML language. In XQuery, path expressions are part of the language, and numeric conversions are automatically done for untyped data. If the data is validated against a schema, the types assigned by the schema are used. This makes it possible to solve the same problem much more simply:

For XML-centric applications, an object-oriented representation of an XML document imposes unneeded overhead that complicates programs.

XML is not just text!

To many intelligent and articulate XML programmers, “XML is just Unicode with pointy brackets” is almost a statement of faith. Predictably, these people also complain that it is difficult to process XML without a parser. For instance, Joe Gregorio notes that in XML this document:

must be treated as identical to this document:

To many of us, this is merely an indication that XML must first be parsed and converted to an appropriate data model before it can easily be processed. In fairness to Joe, he initially assumed this as well, but then changed his mind:

More XML experience is gained by yours truly and on many occasions I have found myself pining for the ability to do regular expression processing of XML. If only the pathologies of the above examples didn’t exist then I could use a combination of XPath and regular expressions to perform XML manipulations that would be easier for me to implement, understand and maintain.

Today I reached the breaking point. The problem isn’t with regular expressions, the problem is with XML. The pathologies in XML that preclude the use of regular expressions are just that, pathologies, and ones that need to be excised.

As a result, he suggests that XML be subsetted as follows:

• All namespace declarations must be done in the root element.
• Never a declaration for the “” namespace. I.e. if an element sits the “” namespace then the element name will never have a namespace qualifier.
• No CDATA sections.
• No DTDs

The above restrictions would make it easier for a programmer to work with XML without using an XML parser, but it is unlikely that the XML community will replace XML with something along these lines – especially since there are important usage scenarios for features like DTDs, schemas, and the ability to build compound documents without knowing, at the root level, all of the namespaces that may be used in a document. More to the point, Joe’s original reason for trying to solve these problems with XPath and regular expressions was that the standard APIs do not make it easy to solve many simple problems. Looking at his article as a whole, and other articles he has written, we believe that many of these difficulties are caused by the same kind of semantic mismatches that a Native XML Programming Language is designed to solve.

In this paper, we assume that XML will remain as is, and that for general processing, the best approach is to use an XML parser to build a data model instance from the XML documents, and query the data model instance. Not everybody believes this is the best approach. Tim Bray, one of the editors of the original XML specification, objects to the Native XML Programming solution because he objects to the notion of an XML data model:

The notion that there is an “XML data model” is silly and unsupported by real-world evidence. The definition of XML is syntactic: the “Infoset” is an afterthought and in any case is far indeed from being a data model specification that a programmer could work with. Empirical evidence: I can point to a handful of different popular XML-in-Java APIs each of which has its own data model and each of which works. So why would you think that there’s a data model there to build a language around?

Tim first says that there is no data model for XML, then argues that there are several. The differences among these data models, while annoying, are not great, and could have been avoided if XML had had a full-fledged data model. The differences between the DOM data model and the XPath data model are well known in the XML world. XQuery, XPath, and XSLT now use one common data model, which can represent both XML and the XML Schema datatypes. Although it would have been convenient if XML had defined a data model, there is no requirement that the data model used by a Native XML Programming Language be the same as any particular data model used in a Java API. As long as the data model supports the structure of XML directly, without losing or adding information in violation of the XML spec, it can be used as the basis for processing.

Tim also suggests that XML is “syntactic”, as though this implies that there is no data model. This implies that syntax and structure are opposites, which is rather surprising, since the purpose of a syntax is to describe the structure of a language. In the XML Recommendation, the structure that corresponds to a data model is called the logica structure:

Each XML document has both a logical and a physical structure. [. . .] Logically, the document is composed of declarations, elements, comments, character references, and processing instructions, all of which are indicated in the document by explicit markup.

Like most modern computer languages, XML uses a BNF to describe the syntactic representation of these structures. For instance, here is a production from the XML Recommendation:

The XML Recommendation is largely a description of these logical structures and the relationships among them. For instance, consider the following text:

Example: The element structure of an XML document may, for validation purposes, be constrained using element type and attribute-list declarations. An element type declaration constrains the element’s content.

Element, XML document, and content all refer to logical structures that are represented in the BNF. These logical structures, taken together with the relationships among them as described in the XML Recommendation, come very close to being a data model, but the data model was not fully described.

The whole point of parsing is to create structures from a sequence of characters, using a grammar to determine which structures to create. When a parser is used to interpret the characters of a program in Java, it creates an Abstract Syntax Tree. When it is used to interpret the characters of XML, it creates a data model instance. We use parsers because (1) the parsed structure is more convenient for further processing, (2) the parsed structure distinguishes information from noise, eliminating differences in the character representation that are not significant in the relevant model, and (3) the parsed structure can fill in information not explicitly represented in the serialized form.

However, an XML parser is not enough. A parser creates a convenient representation of XML. We need a Native XML Programming Language to provide convenient processing of this XML.

What should a Native XML Programming Language do?

A Native XML Programming Language must provide the fundamental operations needed for XML. Some of these operations are required because of the structure of XML itself.

A Native XML Programming Language should be able to easily find anything in an XML structure. XQuery, like XSLT, uses XPath for this purpose. Every XPath expression is also an XQuery expression. For instance, if the variable $cust is bound to a Customers element that contains the rows of a relational table, represented using the SQL/XML mappings, then the following path expression finds all the CustIds from that table:

A Native XML Programming Language should be able to easily create any XML structure. XQuery uses the syntax of XML for this purpose. For instance, the following XQuery expression creates a Customer element:

When XQuery uses the syntax of XML, a curly brace escapes to the syntax of XQuery, allowing dynamic expressions to be inserted. Here is an example that creates a customer with a new unique identifier:

A Native XML Programming Language should be able to easily combine and restructure information from XML sources, operating at the logical level without requiring the programmer to think about the internal representation of the XML. For instance, if we are operating on the SQL/XML views of the customers database, the following XQuery combines customers and projects to show the name of a customer and all projects associated with that customer:

A Native XML Programming Language should be able to easily use XML data in expressions. For instance, arithmetic operations should be able to work directly with XML content, observing the data types of typed data and converting appropriately when they encounter untyped data. It should be able to leverage schemas that have been imported into a query, but work well on XML structures for which no schema has been imported.

In short, a Native XML Programming Language should be able to work with XML the way XML users think of it, easily performing the kinds of tasks that XML users need to have done. XQuery attempts to do just that, based on the usage scenarios we gathered in XML Query Use Cases.

XQuery and SQL/XML Views

Some people seem to believe that the purpose of XQuery is largely the same as that of SQL/XML – to allow XML structures to be created from relational data. Although XQuery is useful for this task, it has relatively few advantages over SQL/XML when this is all that is required. The reason for this is simple: SQL is a language designed for handling SQL data sources, and it does that very well. Adding XML publishing functions to SQL is a simple way to let it create XML. However, it is interesting to note that the SQL/XML views of relational tables have a very constrained structure, and XQuery performed on such views is generally quite similar to the equivalent SQL/XML.

For instance, let’s write an XQuery equivalent to the last SQL/XML query we used. This query will operate on a SQL/XML view of the relational tables. The Projects table is represented as follows:

The Customers table is represented as follows:

We want to rename these elements and create a representation that shows customers together with their projects. The output should look like this:

Here is an XQuery that creates the desired output:

Let’s compare this XQuery to the SQL/XML query from a prior section:

In this example, as in most such examples, it is hard to argue that either solution is particularly superior to the other. Either SQL/XML or XQuery handle such tasks quite well. The real strength of XQuery is in the ability to easily process XML, whether or not relational data is being processed, including the XML that is frequently stored in columns of relational databases and the XML of web messages. Since XQuery also works well on SQL/XML views of relational data, it is particularly useful when both XML data and relational data must be used in processing. This is explored in the next section.

Spanning Sources: XQuery, Web Messages, and Databases

XQuery, when combined with a SQL/XML view of a relational database, is extremely good for processing XML together with relational data. This is a very common requirement in many environments, including web message processing environments. To illustrate this, we will use Example 1 from the SOAP Primer. The task is as follows: an incoming message requests a flight to Los Angeles departing from New York as follows:

According to the SOAP Primer, the proper response is to point out that there are three airports that depart from New York, so that the user can be prompted to pick one. Here is the desired output:

Reading between the lines, we assume that there is a database somewhere that lists the airports for each city. The SQL/XML view of the airports table might look like this:

We will assume that when there is only one airport for a city, the output should simply list that city, and that an error should be raised if there is no airport for a given city. The following XQuery handles all three of these cases:

Note that this code operates at a level very close to the application domain, rather than navigating XML documents and converting from XML to appropriate types in the host language. XML data sources and relational data sources are treated in the same way – to the query, they both look like XML documents.

XQuery for Java (JSR 225)

SQL programmers are used to using APIs such as ODBC or JDBC to set up the environment, execute queries, and do processing in the business domain using the data returned by a query. Similar APIs are expect to emerge for XQuery. The first standard API for this purpose is now being developed under Java Community Process. It is known as XQuery for Java (XQJ), or JSR 225.

Significantly, the requirements of JSR 225 ensure that both XML documents and XML views of databases will be supported, and the results of a query can be processed using JAXP and SAX

SQL/XML and XQuery: Do we need both?

Although SQL/XML and XQuery are both XML query standards, they are based on quite different models, and fit best in different architectures. SQL/XML fits cleanly into the relational model as a reasonably small extension to traditional SQL. This means that it works well in traditional SQL environments, providing full access to the existing SQL language, including features like updates and full-text queries that are not going to be part of XQuery 1.0. One of the other advantages of using SQL as a basis is that database manufacturers have many years of experience in optimizing SQL queries, which means that many of the optimization issues are well known. Also, it has existing APIs, including ODBC and JDBC. In short, SQL/XML provides the functionality needed for creating XML from relational data while still fitting cleanly into the existing SQL environment. SQL/XML implementations will be available from Oracle and IBM, but not Microsoft, and a cross-database implementation is available from DataDirect Technologies. Oracle’s implementation also provides functionality for querying and processing XML as well as SQL, and there is some interest in adding extensions along these lines to SQL/XML. Some members of the SQL/XML task force would also like to see parts of XQuery added to SQL/XML.

XQuery fits more cleanly into the XML environment, providing Native XML Programming for both XML sources and non-XML sources accessed via an XML view. It is well designed for combining data from multiple sources, and is very efficient for a variety of XML programming tasks. However, XQuery is a brand new language – in fact, at the time of writing, XQuery 1.0 is merely a Working Draft, not likely to emerge until the second half of 2004. There is a great deal of enthusiasm surrounding XQuery, most major database vendors have announced support for it, and there is a great deal of research on optimizing XQuery. However, XQuery is a much younger language, the industry has little experience optimizing it, and it lacks some features, including updates and fulltext, that are very important for some kinds of tasks. Also, the API for XQuery, XQuery for Java (JSR 225) is just now being developed.

Oracle Text uses standard SQL to index, search, and analyze text and documents stored in the Oracle database, in files, and on the web. Oracle Text can perform linguistic analysis on documents, as well as search text using a variety of strategies including keyword searching, context queries, Boolean operations, pattern matching, mixed thematic queries, HTML/XML section searching, and so on. It can render search results in various formats including unformatted text, HTML with term highlighting, and original document format. Oracle Text supports multiple languages and uses advanced relevance-ranking technology to improve search quality. Oracle Text also offers advanced features like classification, clustering, and support for information visualization metaphors.
To make it simpler to see what to do, let’s set up an example package with a similar interface to CTX_THES.  This example ctx.sql just returns random data:

-- Package "SuppliedPkg" simulates Oracle Text's CTX_THES.
-- It has a procedure that returns a PL/SQL type.

create or replace package SuppliedPkg as
  type SuppliedRec is record (
      id   number,
      data varchar2(100)
  );
  type SuppliedTabType is table of SuppliedRec
       index by binary_integer;
  procedure SuppliedProc(p_p in out nocopy SuppliedTabType);
end SuppliedPkg;
/

create or replace package body SuppliedPkg as
  procedure SuppliedProc(p_p in out nocopy SuppliedTabType) is
  begin
    -- Create some random results
    p_p.delete;
    for i in 1..5 loop
      p_p(i).id   := i;
      p_p(i).data := 'Random: ' || i ||
               (1+ABS(MOD(dbms_random.random,100000)));
    end loop;
  end SuppliedProc;
end SuppliedPkg;
/

Run it in SQL*Plus like:

sqlplus hr/hr@//localhost/XE @ctx.sql

This is the “fixed” part of the problem, representing the pre-supplied functionality.  It seems impossible to call SuppliedProc() and return its data to PHP.
Since you can’t change SuppliedProc(), you can create a helper function in PL/SQL to convert the PL/SQL type SuppliedTabType to a pair of SQL types. Myproc.sql is:

-- Create a wrapper procedure that calls the pre-supplied
-- SuppliedProc() and converts its output to SQL types.

create or replace type MyIdRec as varray(100) of number;
/
create or replace type MyDataRec as varray(100) of varchar2(100);
/

create or replace procedure MyProc
      (p_id IN OUT MyIdRec, p_data IN OUT MyDataRec)
as
  l_results SuppliedPkg.SuppliedTabType;
begin

-- get results from existing procedure
  SuppliedPkg.SuppliedProc(l_results);

-- copy to a type we can pass back to PHP
  p_id.delete;
  p_data.delete;
  for i in 1..l_results.count loop
    p_id.extend;
    p_id(i) := l_results(i).id;
    p_data.extend;
    p_data(i) := l_results(i).data;
  end loop;

end MyProc;
/

Use SQL*Plus to run this script too.  Now you can call MyProc() in ctx.php:

size(); $i++) {
  $id   = $res_id->getElem($i);
  $data = $res_data->getElem($i);
  echo "Id: $id, Data: $data";
}
?>

This allocates two collections and binds them as the parameters to MyProc().  After MyProc() has been called, the collection method getElem() is used to access each value returned.

The limitation of this example is that you have to set an upper bound on the MyIdRec and MyDataRec VARRAY sizes.  Simply replace both uses of VARRAY2(100) with TABLE to overcome this.

function pageCallback(response){

alert(response);

}

var ajax = new AJAX();
ajax.callPage(“mypage.html”, pageCallback); serviceCall example:

function serviceCallback(response){

alert(response);

}

var ajax = new AJAX();
ajax.callService(“MyWebService.asmx”, “MyMethodToCall”, serviceCallback); serviceCall with parameters example:

function serviceCallback(response){

alert(response);

}

var ajax = new AJAX();
/* paramters are added to the end of the callService method in key=value pairs. */
ajax.callService(“MyWebService.asmx”, “MyMethodToCall”, serviceCallback, “myParameter1=hello”, “myParameter2=world”); serviceCall with custom namespace example:

function serviceCallback(response){

alert(response);

}

var ajax = new AJAX();
/* set new namespace for use with web service. */
ajax.setNameSpace(“http://mynamespaceuri/”);
ajax.callService(“MyWebService.asmx”, “MyMethodToCall”, serviceCallback, “myParameter1=hello”, “myParameter2=world”); error handling example:

/* by default the errors are alerted. To suppress this you must redirect them. */
function myErrorHandler(error){

alert(error);

}

var ajax = new AJAX();
ajax.onError = myErrorHandler; Download Download the latest version at http://sourceforge.net/projects/glm-ajax/  
]]> http://gudangberas.wordpress.com/2006/08/10/ajax-framework/feed/ 1 gudangberas http://gudangberas.wordpress.com/2006/08/10/oracle-database-for-debian/ http://gudangberas.wordpress.com/2006/08/10/oracle-database-for-debian/#comments Thu, 10 Aug 2006 08:45:41 +0000 gudangberas https://gudangberas.wordpress.com/2006/08/10/oracle-database-for-debian/ ]]> Debian Packages: libc6-dev binutils

Installing Oracle on Debian has involved obtaining the Oracle tar file, setting up an Oracle user and some Oracle groups, and running the Oracle installation software. We include here a collection of information for installing Oracle 8 and Oracle 10g, with some pointers regarding Oracle 9i. The good news is that Oracle now (as of March 2006) have an apt repository for the unstable (sid) distribution, and have provided the Oracle XE packaged for Debian. It is available from http://oss.oracle.com/debian/.

Additional information is also available from http://openacs.org/doc/openacs-4/oracle.html, http://frits.homelinux.com/wordpress/?p=9, and http://www.davidpashley.com/articles/oracle-install.html.

To install Oracle XE you first need to tell wajig where to obtain the packages from:

$ wajig editsources
      -> deb http://oss.oracle.com/debian unstable main non-free

Then install the libaio package–since version 10 of Oracle on GNU/Linux asynchronous IO has been the default “out of the box.”

$ wajig install libaio

Next, install the appropriate Oracle package. If you want just a client, then:

$ wajig install oracle-xe-client

To install the usual Oracle XE server:

$ wajig install oracle-xe

And to install a version of the server that supports needing multibyte character sets inside the database:

$ wajig install oracle-xe-universal

Subsections

• Oracle 8
  • Obtaining the Distribution
  • Oracle User and Groups
  • Oracle Installation
    • File Locations
• Oracle 10g Release 2
  • Preliminary
  • Getting Oracle 10g
  • System Parameters
  • Oracle Accounts
  • Kernel Parameters
  • Oracle User Setup
  • Installing Oracle 10g
  • Starting and Stopping Oracle 10g
  • Using Oracle 10g
  • Importing data from Comma Seperated Value (CSV) files
  • XDB
  • Troubleshooting
    • Missing Packages
    • X DISPLAY
    • runInstaller as oracle
    • Starting iSQLPlus
    • NLS DATE FORMAT
    • Enabling Swap

Reliable

MochiKit has HUNDREDS of tests. We build real applications with this thing. So even though development can move fast, we make sure to get tests written. This also makes platform compatibility issues much easier to detect and resolve than the “guess and check” style of quality assurance seen in some of the other libraries out there. It’s not broken.

Documented

You’re unlikely to find any JavaScript code with better documentation than MochiKit. We make a point to maintain 100% documentation coverage for all of MochiKit at all times. You don’t have to fumble around reading our source code or leafing through examples to find out how something works.

Evolutionary

MochiKit can adapt to anything you throw at it. It makes no assumptions about how your code needs to act, and it has hooks (by way of the the adapter registries) that makes sure that you can define your own comparisons, programmer representations, iterators, or DOM node coercion for any object in any way you wish. We’ll gladly serve you all the Kool-Aid you want, but we’re not going to make you drink any.

Plays well with others

Although MochiKit can be, and often is, used as a one stop shop for JavaScript goodness; it is very highly interoperable. MochiKit does no Object.prototype hacking, and inserts just three symbols (the MochiKit namespace, plus the compare and reduce functions to work around JScript bugs) into the global namespace. Unless, of course, you ask it to bring in more symbols for convenience.

As far as server-side goes, MochiKit is totally agnostic. However, feel free to include MochiKit as part of your AJAX arsenal – TurboGears does out of the box!

All your JavaScript are belong to…

The new packaging systems that have sprouted up recently are extremely cool, and WILL be revolutionizing JavaScript development. JSAN and Dojo Toolkit are fully supported by and interoperable with MochiKit! Don’t worry though, MochiKit still works fine on its own if you aren’t interested in those new fangled packaging systems.

Free as in…

You won’t see any political agenda nonsense here. MochiKit is available to you under the extremely liberal MIT license, or the written-by-a-very-smart-lawyer Academic Free License, v2.1. It’s your choice! We don’t want to “infect” YOUR software with our ideals, we just want to make every developers’ life suck just a little bit less. MochiKit is free as in we don’t care what you do with it. With the choice of these two licenses, you can put MochiKit anywhere. We certainly won’t refuse if you offer free beer, though.

Compatibility

Our current test platforms include all of the modern and popular browsers: Safari 2.0.2, Firefox 1.0.7, Firefox 1.5b2, Internet Explorer 6, and Opera 8.5. Other JavaScript platforms should work if they’re standards compliant (ECMA-262 and the W3C DOM ECMAScript binding, primarily).

Yes, we’re aware that a lot of people use other browsers, but the above list matches an astronomically high percentage of the intended audience of our flagship application. Please let us know if you encounter any problems elsewhere, especially if you can contribute a fix.