Archive

Archive for October, 2010

Project Added – ‘PV-Spiders’

October 12, 2010 1 comment

I feel like I’m getting a bit nostalgic all of a sudden.. 🙂

PV-Spiders

This is another product I created for Prime Vendor in Wilmington, NC in order to collect all the bid information offered by any government agency in the United States.

The core problem here was that there’s somewhere around hundreds of thousands of organizations in the United States government that post bids for private contractors to, well, bid upon.  Many of these organizations post multiple new bids every day – some of which are streamed out of a database but, surprisingly, lots of these government agencies still relied on human beings to update their tables of offers.  When your job is to create a network of ‘spiders’ that go to these pages and are just supposed to check if anything has changed since their last visit and, if so, download the new data and submit it as a new bid.. well, there are often complications.

All the people before me faced with this problem created programs that’d establish HTTP requests to these sites and then attempted to parse the html text that came back in a meaningful way where they could then either download the bid or continue on to the next stage of the website until they can, hopefully, inevitably get to the direct bid file.  Many sites required you to login and provided hundreds to thousands of results over the course of several html pages – and many others were almost entirely javascript which these spiders just couldn’t go to.

Being a little lazy, I decided to forgo all of this nastiness and instead chose to extend the built in .NET WebBrowser control to the point where you could register parsing events to given Uris and, whenever the browser control would hit one of these Uris, the event would be called and you’d have the entire page parsed for you already in the form of the DOM (Document Object Model).  You could then use Linq (or old-school for-loops, should you prefer) to extract out whatever information you needed from this page in a handful lines of code.  You could even execute JavaScript.  This entire overhaul of the WebBrowser control took about 50 lines of code and less than an hour to develop.

(click for more)

Advertisements

Project Added

October 11, 2010 1 comment

It appears that boredom can, in fact, strike twice in the same day – I’ve added another entry into my projects section for anyone interested.

Commerce Network

Commerce Network is a product I created for a small Wilmington, NC – based company by the name of PrimeVendor Inc. It was designed to electronically handle all invoices, purchase orders and receiving reports passed between the private sector and the United States government in addition to all state governments governed by the US.  At the time of creation it was typical of most government agencies (and many contractors) to handle all of this manually on paper through snail mail and faxes with humans processing the documents.  Needless to say, the cost savings of replacing all of these humans and paper with a software, paperless solution are dramatic and compelling.

In order to meet these lofty goals the product had to be designed in such a way as to support any and all delivery mechanisms, protocols and data formats and, similarly, be able to then transform this data of ‘any’ format to an output destination of ‘any’ protocol in whatever format the receiver required.  It also had to be able to scale to the levels of potentially handling billions to trillions of such interactions in a day.  Additionally, we needed a web portal accessible by both our clients and the agencies they dealt with where the users could create, send and receive these documents in addition to being able to view the logs of all documents that passed through our system (and bring up each archived document for display).

(click for more)

New ‘Projects’ Section, Theme

So it appears that I finally got bored enough to update my blog with a new theme and a projects section detailing some of the current and past projects I’ve worked on.  For now I’ve just added my current project, MarginPro, but as boredom ensues I’ll add some of the others..

MarginPro.com

MarginPro is an extremely extensive cloud (Windows Azure) application consisting of probably the single most fully featured and beautiful business application written in Silverlight 4 currently around.  In addition to this there is also our public facing MVC application that hosts all of our marketing material, login page and a full featured help (editable and expandable in real-time by admins) and support site complete with training videos, tutorials, the ability to log support cases directly to our support department and numerous deep integrations with our CRM system to enable our sales and front-line support staff to handle the vast majority of issues that arise without ever needing to expend any development team effort.

This project is a perfect example of the benefits of software done right – if any of our customers any where in the world ever encounters an error, our development team is instantly notified via extremely detailed e-mail complete w/ the error message, stack trace and a few other pertinent bits of information.  Usually within a matter of minutes we’re able to resolve these issues and can then, with the click of a button, deploy the fix to every single user of our system without even a second of downtime or a single user interrupted.  Likewise we’re able to develop, test and deploy new features the same way with a turnaround rate of ‘several’ to a dozen+ features pushed to customers every week.

 

(click for more)