Intro

These are details of various projects, big and small, that I have worked on over the years. This is only a list of a few. Of course there are many more.

PASCARGOUSA.COM

This is a website for a shipping company.

1. Standard company pages
2. Incorporates a live help chat system
3. Quote form allows customers to request quotes
4. Uses custom PHP template system
5. Direction map includes company location tag
6. YUI manubar

Design

Site designed with customer. Started off with static pages then had to create templates because changes on common page items were taking too long.

Challenges

Started off with Yahoo map. That was successful but started to misbehave in production. Switched to Google map.

Had to create custom graphics for client which was time comsuming. Have gotten great feedback though.

YEAHMAP

This is a Javascript library to create a map on a web page with some specific characteristics.

1. Puts a main marker point on map
2. Puts a secondary marker point on map
3. Main marker is a smart marker with additional pop up HTML markup.
4. Secondary marker is a simple marker just to indicate the starting point.
5. Draw a multi-segmented line from secondary to main marker.
6. Dynamically loads the map api
7. Select the initial map type
8. Optionally add the pan control
9. Optionally add the long zoom control
10. Optionally add the map type control

When the Javascript for the library is loaded, it creates a global object called YEAHMAP. There can only be one map on the page.

This library uses a Yahoo application ID. You should get your own and replace it before using the object.

Design

In development.

Challenges

Yahoo Avatar Viewer

This application was part of a prototype for a potential future mobile application. The intent was to allow mobile users to send their avatars to each other using their phones. This protptype was used to establish understanding of the base functionality of the Yahoo avatar system.

1. User enters yahoo id
2. Application displays all versions of Yahoo Avatar
3. Uses undocumented Yahoo Avatar API
4. Uses RFC822 to display data images in Internet Explorer.

Of particular interest in this application is the use of a little known technique to display data encoded images in Internet Explorer. One of the avatar images returned is an encoded string of the image. Most browsers support the 'src="data:..."' feature which allows them to display the image directly. IE does not support this feature. This uses IE's support of RFC822 to embed images in an HTML file. The resulting HTML file is more like an email. Required use of some scarcely documented techniques using VML, MTHML to package the data.

Design

The only design element to this is to handle the different output based on browser type.

Challenges

This technique uses some undocumented features of IE and it does not behave as one would expect. It took some trial and error to get it to work right.

BOSS Java Library

This is a Java library to simplify use of Yahoo BOSS. BOSS is a webservice that is used to build your own search engine. This library lets you specify any two sources of RSS or BOSS feeds and combine those together in a simplified database-like way.

1. Select two data sources
2. "Join" the the results in a flexible manner.
3. Automatically handle the XML parsing.
4. Treats results like a simplified table.

Design

Challenges

MIT 2.0

In the early days of the First Data gift card system, clients were implemented into the system manually. There was a skeleton SQL file which was modified. At one point, I hired a guy named Mark to do this task along with some other house keeping activities. Mark, being a bit entrepreneural made himself a tool with Visual Basic to make this job much easier. He called his tool Mark's Implementation Tool or MIT. Now this tool was anything but perfect. It was for him to do his job easier. And it requireed doing several steps in a very specific manner. More importantly, it allowed direct manipulation of almost all the important tables. This had several ramifications. It required the implementation team to know the table structures well. This meant longer ramp up times for new people. It also potentially meant more errors.

I pushed for creation of a new tool which would do these things at a higher level. the idea would be to gather information in a wizard like manner and build the necessary data behind the scenes. This we called MIT 2.0. We took extra care to detail out all the possible values that had to be correct. It implemented a mechanism for automatically generating merchant numbers.

1. Implemented intelligent wizard
2. Took a higher level approach to the database.
3. Implemented strong business logic checks for data entry.
4. Required login for enhanced security.
5. Implemented systematic controls for conventions.

Design

This was a plain VB program. It had some direct data screens and a wizard for doing the main implementation.

Challenges

It was necessary to carefully document all the business rules and enforce them in the program.

FEXCO

This was a project to add a way to have a reseller program within the gift card system. The reseller had to be able to do the implementation themselves. It had to be relatively simple and only provide some limited options. This was a total team effort.

1. Resellers could implement their own clients using a web application.
2. Options for these clients were pre-determined.
3. A billing file (XML) was produced and sent to the reseller so they could bill their clients as they saw fit.
4. Devised a system whereby the reseller customer service could access only their clients' accounts.

Design

I designed a system whereby a set of data was created as the model for an implementation. This would allow the model to be changed at a later time without needing any code changes. The final design allowed the reseller to choose one of three types of programs for their clients.

Challenges

The main challenge for this one was not technical but time. There was tremendous drive to get this done before the annual peak season lock down. It became necessary to tightly manage this project features and progress. The development team spent two weeks where they slept very little. The project was delivered on time but there were several bugs which were not fixed until the following release.

FDGCS

This was an enormous project as it involved several business teams, two separate billing departments and the gift card team. Up to this point the gift card system only allowed a very expensive and totally flexible way to implement clients. The goal of this project was to be able to sign up really small clients. The goal was to get 60,000 merchants within two years. Most of these merchants were already clients for other services, mostly credit. The billing for this had to be driven into the existing billing system. This was unusually complicated because of the internal reseller business structure.

I did no coding on this other than some prototypes to establish a proof of concept.

1. Implementation had to be done in minutes and had to be very easy.
2. Merchants were allowed almost no flexibility in how their gift card program would work.
3. Billing had to be done on the existing mainframe billing system.

Design

Because we needed to select default values for just about everything, I designed a set of data items that would determine how the merchant got implemented.

Challenges

This project had numerous challenges. First and foremost, the various teams had never worked together before. The business proceses and technology were entirely separate. After weeks of project meetings, there seemed to be stalemate. No one wanted to take a shot at what the implementation would be like. I broke this stalemate by writing a white paper that drew a line in the sand. In this paper I detailed out exactly what would happen and how it would all work. I knew full well that some of it would change. That provided the business team with some sort of solid footing. Over the next 2-3 weeks, discussions continued. However, I continued to drive them to the document that I provided and amended it as we went along.

The clients were implemented using a mainframe application. We did not want to make the implementation team have to use a separate application. I wanted to publish a web service that the mainframe could use to do the implementation of the gift card program. In the interest of time and some technical limitations, we ended up with the team having to double enter some data. It was not until some time later that this was replaced by a web service interface which, as expected, significantly reduced implementation errors.

Implementing billing was very challenging. Getting a Java/Oracle application to talk to a mainframe/cobol application was areal chalenge. We ended up needing to create some Java code to write a binary file containing cobol packed data.

Giftcard Ticketing

This is a Java based application for streaming data from the First Data Gift card system out to certain outside entities like Disney, Universal Studios and Apple iTunes. The scanario for the theme parks involves customers purchasing a ticket, which is actually just a gift card. They buy this at a store like Walgreens or Safeway. They drive to the park and can immediately use the ticket to go through the turnstile. When they buy the card, the normal gift card transaction occurs. The online gift card application recognizes these cards as tickets and puts an entry into a queue. The ticketing application scans the output side of the queue and processes the entries. It takes the data and creates special XML based on where the card is going. The XML is handed off to a handler which sends the information to the ticket provider(the entity receiving the ticket activation transation).

1. Produces XML based on who the data is being sent to using an adapter pattern.
2. Each ticket provider can have different XML.
3. Typical processing for a transation end to end is a 2-3 seconds.
4. Is smart enough to send transactions for any individual card number in sequence.
5. Multi-threaded handlers send to providers avoiding bottle necks
6. Totally decoupled from online processing enabling a high degree of scalability
7. Very fault tolerant using a retry strategy

Design

Challenges

Design Order Tracking

The business made some good money managing card orders for clients. They provided a one stop shop. This was an extremely manual process and was run using Excel and email. Each representative handled their clients as they saw fit and there was little standardization. They hired a consulting company to analyze their business. This company recommended a workflow application to manage all the steps.

1. This was mainly a work flow application
2. It had to implement business logic of the card ordering process.
3. Had to produce invoicing data.
4. Produced several reports.

The cosulting company was given the go ahead to build this app. After some time, they determined that they had done the best they could and wanted to walk away even though the application was not completed. I was asked to evaluate this application to see if I could help, offer a recommendation or somehow get it to be successful.

Design

The core of the application was the Savvion Business Process manager. This was a data driven workflow engine. The consulting company had implemented numerous enhancements and custom code to get the application to do what the business wanted. Much of it was not implemented. Much of this we learned as we went through the code.

The one major missing component was a reporting sub system. There was no way to do reports out of the existing database. It just was not structured fir that. So I designed a reporting structure based on the report requirements. I then created an extraction tool that took the data from the main database, transformed it to the reporting structure and loaded. We then had JReports as the reporting tool which we used to build the reports required.

Challenges

This project was challenging on many levels. One of the first challenges was that it was a memory hog. I worked for some time tweaking the Java Virtual Machine runtime parameters which helped. Eventually, what helped the most was a change to the way Hibernate was implemented. A few code changes resulted in a huge improvement.

It became clear that the requirements were never clearly described and the original developers had implemented what they thought they needed, not necessarily understanding what the business needed. The business was fragmented. Since they handled their clients differently, it was not clear what the requirements should be in some cases. The technology was a mess. The database schema needed to be redone but there was no funding to do it. My strategy was to get it stable to the point where it was usable. We recorded all the bugs and missing features and continually chipped away at these. It eventually became clear that this product was not viable. Over the course of the build, the business had changed to a degree. They had cleaned up some of the anomalies that had lead to the decision to build the product in the first place.

As we went through the code, we found more and more missing functionality. There were IF statements with nothing to execute. There were on screen fields with no corresponding database entries or code in some cases.

Card PIN Encryption

This was a part of a bigger project. This full application produced manufacturing files for gift cards. Part of this process involved generating a PIN. the front end was required to allow the application user to enter a card and PIN and verify that the PIN is correct. This was an internal application and the verification was needed as part of the QA process. Once the cards were made, any changes would require huge sums of money so accuracy was very strongly desired. This base process used a set of keys and 3-DES encryption to produce the final PIN. I wrote a library in Java for use by the front end application. This required deconstructing the C code which used the OpenSSL library in a unconventional way and building the exact same functionality in Java.

1. Used Java PKI encryption
2. Used 3-DES
3. Used SHA-1
4. Required bit manipulation

Design

Created a top level class that would be instantiated. Then a series of calls are made to methods to validate the PIN. Once everything was encrypted, a socket call is made to the Giftcard encryption service.

Challenges

The C code used various bit shifting techniques which had to be duplicated exactly in Java. There were some issues with shifting bytes in Java which are alyways signed. It was difficult to match the algorithms and steps in the C code to the Java equivalents. Starting with no experience with the Java encryption libraries made the process longer. Debugging calls to the encryption service was difficult because it did not properly log error conditions. Therefore, when something failed, there was not adequate information to know why.

System Stats Tracker

This process was used by the operations center to track statistics on a daily basis for reporting to executive management. The process scanned logs on each of the boxes and rolled the information up to a single box where the information was combined into a total number. The individual parts of the process ran automatically and pushed their information up to the central location. The second part was run manually when the operator ran a script to gather the information for posting to the a central database.

1. Scanned log files for transaction counts.
2. Uploaded stats from each individal box to a central location.
3. Consolidated the transaction counts for executive management reporting.

Design

The overall process was split into individual processing and overall processing.

Challenges

This was actually a fairly simple process.

Promo Code Expiration Updater

This was a back end process that was used on an ad hoc basis. In some cases, the expiration date had to be modified on a huge number of gift cards. This process did this update. While doing the update was not a very difficult thing, the real challange to this was to keep processing under a certain threshold so as not to impact the overall live system processing. If the process died or was killed intentionally, it had to be able to know where it was and pick up from there with no errors. It had keep progress and log everything it did.

1. Could not exceed a certain number of transactions per second.
2. Had to detect if a commit was taking too long and end the process if detected. This was to ensure that if there were any database processing throughput issues, that this process was not causing it.
3. Had to log everything it did.
4. Had to be able to pick up from where it left off.

Design

This was a faily straigtforward process and did not require much design. It used PRO*C to enable the most efficient processing. Using a database cursor the program worked on chunks at a time. This value was configurable but typically done in 1000 at a time. Once the 1000 were done and committed, the cursor was updated and a new range was acquired. Program measured how long the commit took and ended all processing if this exceeded two seconds.

Challenges

It was important to be efficient and this meant processing ranges at a time. It would have been inefficient to execute SQL for one record at a time. Equally bad would have been to try to update everything at once.

TPS STATS 2.0

Each year during the holiday season, the business team and the clients wanted continuous reports on transaction volumes. Initially a simple PERL program was created that did this. It had some hard coded values for a few merchants. As more clients were added, this became a problem adding the clients. I redesigned and rewrote the application to use a configuration file and a single job that ran every two hours. This would attach to the application log file, sample a minutes worth of data and send out a report via email. Email was done to email groups which very often were sent to pagers.

1. Captured client stats
2. Sent out stats via email

Design

I created a global configuration file that determined how to identify each merchant's data in the log and where to send the summarized results.

Challenges

It was important to be able to change who received the reports easily. Therefore, the email was sent to a mailing list. The mailing lists were managed by the business. This meant that no changes had to be made to production in order to change who got the reports.

Minibert

TPS is Transactions per second. This is one of the most important measurements of this system. At various times, the giftcard system was only capable of handing a certain level of throughput. And it was important to watch the system to see if it was approaching that threshold. I created an application using PERL and CURSES that continually scanned the logs and reported the transactions per second. It also kept track of certain thresholds and looked for certain errors.

1. Monitored TPS for the system
2. Monitored various errors
3. Used by operations to easily monitor the system.

Design

This application was built with PERL. It scanned a log file. It had to parse the log file contents on a real time basis and keep track of statistics, counts and averages.

Challenges

The application was not able to switch over to the new log file when the new daily log file was created. PERL did not have stable multi-threaded capabilities. It was necessary to restart the application when it crossed the midnight boundary.

TPS High Response

This was a report that ran every day looking for transations that took too long to process. This typically meant that the transaction took more than 2 seconds to process.

1. Simple report looking for high reponse time transactions.

PTD Process

This was part of a back end reconciliation process for gift card processing between First Data and some of its gift card clients. This PERL application was designed to be run once a day and extract all the transactions for a specific client for the previous day. The file produced was converted to a mainframe format when it was uploaded using FTP. The client ran a process to ensure that their records of what was processed matched with First Data. Within two weeks of this process being put in place, a fraud situation was uncovered.

1. Extracted transactions from transaction table
2. Created export file
3. Sent file to mainframe for sending to client

This became a standard process for all clients.

Billplex

Billplex was a billing product for cell phone carriers. It took in usage data and applied rate plans for the generation of customer billing. I worked on this as a coder, designer and team lead from 1996 - 1997. I had two developers on my team. This project was substantial. It was separated into three parts.

1. Front end
2. Back end
3. Reporting
The front end was created using Delphi. That was done by a another team. The back end was done in C++. This was my team. The reporting team was done by another team.

Design

I was the team lead for the back end development. This was the part that processed the usage data and produced billing data. We determined that we needed some sort of custom logic capability. This was to allow each installation to customize the various parameters that drove the billing process. We considered creating a scripting language and then decided to build drivers and use C++ as the scripting language. This was fine as there was no intention to allow outside entities to create these drivers anyway. The billing process ran in two steps. The first step scanned all the items and applied billing logic to those items. The next stage was Aggregate Billing which applied logic to the bill as a whole. This was how we implemented things like "first 100 minutes free".

The back end had a socket based server application that listened for commands. It would then execute the commands. I designed a multi-threaded structure where various worker threads would be waiting for jobs to do. I wrote the dispatcher which took the incoming commands, found a worker that could handle the type of request, send it to the thread and send the results back when it was done.

I wrote several sample drivers for testing and then wrote the first driver for the first client. It turned out to be really easy.

Challenges

Getting the multithreaded application to work was challenging. And the threads did not balance out. In retrospect, that did not really matter.

AT&T - Siemens Switch Data Converter

I worked on this project as a designer and coder in 1994-1995 timeframe. This was a C++ application that ran on SunOS. I worked on this with another developer.

This application aided in the conversion of telecom switch data from ancient AT&T 1AESS switch to the Siemens EWSD. The application had two functions.

• Load the 1AESS data and output EWSD data.
• Compare the 1AESS data with the EWSD data.

Design

The application had to be able to compare the data from both switches. These were very different types of equipment and the data showed that. We needed a common internal format so we could match like with like. We settled on an AVL tree as the main storage structure, using the phone number as the key. This worked out well. I designed a loader mechanism using the adapter pattern. Each type of data being loaded would have its own loader which would populate the common data model. I built the loaders for both switches and the other programmer dealt with the comparison function. We also used an adapter pattern for output. That would let us output various formats by just adding a different output driver later on. The output drivers and loaders were created as dynamic libraries and loaded at run time.

Challenges

Perhaps the biggest challenge was the representation of two types of data that had many similarities but were not exactly the same. This made the comparison code need many custom sections to deal with all the anomalies.

The input data for the 1AESS was very tangled and it took quite a bit of effort to normalize it to fit into the data model.