Continuum Photography

I wanted to take a moment to share with everyone my sister and brother in law’s new, absolutely phenomonal, photography company - Continuum Photography.

As long as I’ve known Josh I have never once seen him without a camera in his hand for more than five minutes.  They are both extraordinarily talented (in both photography and post processing) and dedicated.  They do weddings, portraitures, special events, commercial photography and more - basically anything you’d want a professional photographer for.  They’ll travel just about anywhere in the world for your events, are highly professional, have wonderful personalities and are determined to ensure your satisification with their services.

Thiefs, Silverlight and more

First of all I want to get something off my chest - STOP STEALING MY NICKNAME!  When I first started using the ‘Ilfirin’ nickname a little over 10 years ago you could put it into any search engine and always, without exception, receive 0 results.  It remained like this (with the exception of various papers, articles and posts I wrote around the internet) for a good 3-5 years, but now if you google ‘ilfirin’ you’ll receive somewhere around 85,000 results (maybe 5,000 of which are me).  What the hell?  It’s not exactly a ‘normal’ nickname.  I’m not even especially fond of it anymore but, damnit, I had it first! 

Anyway..

 

As far as my graphics R&D work goes, not too much has changed since my last post (where I mentioned nothing would be changing for quite a while!).  I have, at least, resumed my R&D work on real-time BRDF rendering and I’m still making a good deal of progress.  Alas, I only have 2-4 hours a week to dedicate to this and that’s generally not even enough to get the math right for one section of what I’m working on.  Plus, something “super secret” has popped up regarding everything I’ve been working on (the highlight of which is the BRDF stuff, although there’s about a dozen different projects I’ve been working on included in it) that’ll prohibit me from talking about any of this in too much detail until said “super secret” project is commercially released.

[Insert Transitional Phrase]

Silverlight! 

So I’ve been steady at work at my new job writing their entire software suite from the ground up for next generation platforms (as the original code base was written close to a decade ago in VB6) one of which we’ve decided to write in Silverlight, despite it’s extremely infant state.  We were originally doing all the web application parts with ASP.NET and AJAX and have some stuff that looks and runs absolutely amazing for those platforms (relative to everything else I’ve seen developed on those platforms).. but it’s still fucking horrible.  If you want to write applications that have completely random, unexplicable bugs and have functions that *should* only be called once suddenly called 20-30 times for no reason (not even counting postbacks and callbacks), half of the time losing the data to that call in between the call and the actual procedure’s code.. write for ASP.NET and AJAX.  Likewise if you want some interfaces that are so fucking close to being awesome but that tiny little bit it falls short, is just enough to make it absolutely horrible.  There are tons of things asp and ajax are good for, writing what are essentially windows thick clients to be run in a web/browser environment, on the other hand, is not one of them.  Do yourself a favor and avoid them like they’re going to give you herpes.. because they might just cause you to do stuff that will!

Silverlight on the other hand.. well, to regress, I was actually one of the ones adamant against using Silverlight.  I kept beating the horse saying that it wasn’t mature enough (this one is still largely true, but read on), it wasn’t for doing the type of applications we needed/wanted to do, it might not even be supported in a few years and on and on.  Mostly I already had done extensive work on the ASP.NET and AJAX applications and finally had all them working like I wanted and really didn’t want to spend a month working with a technology I knew nothing about that quite likely just wouldn’t work for what we wanted to do.  Luckily, I lost that fight.  It took me less than 2 full work days to get the basics of our application working in Silverlight and, amazingly, we ran into no bugs.  The platform is just SO much more solid than any other web platform out there right now.  Once you have everything working the way you want it to.. it stays working.  Probably the most important difference between Silverlight and ASP/AJAX (not that they’re even direct competitors, it just happened to be an either/or situation *for us*) for us was in handling the user interface and layout.  For our application we wanted it to always make full use of the entire browser window (with a bunch of collapsible sections in the application) no matter how big or small it is and never require scrolling even when it only takes up a small chunk of the screen.  Getting together a proof of concept of this running in ASP/AJAX was actually extremely [misleadingly] easy.  We got that together so quickly and so close to what we wanted that we initially abandoned all research into other areas and just decided to go this route.  Huge mistake.  When we started making the necessary corrections to the built-in controls for our needs all of a sudden all our layout code (that you’d think/we thought would be 100% seperate) started going completely nuts.  Change out a textbox in a grid to a combobox in the same grid and all of a sudden the grid would shoot out a few thousand pixels off the side of the screen filled with nothing but empty space.. to name one of about 3 billion instance of strange, completely uncalled for problems.  All the sections of the application that were developed and working completely fine in isolation were suddenly broken in inexplicable ways when moved into the main application despite the fact that there was exactly 0 overlap and 0 communication between them.  Whatever - I don’t even want to go into half the problems we ran into with all this (although I would LOVE to know why everything worked perfectly fine in the R&D stage but completely bombed in production!!!!)

Silverlight isn’t without it’s faults, however.  In fact, in the current beta release (2.0 beta1) there’s some completely fucking retarded parts to it.  Quite a few actually.  For one, there’s no buit-in ComboBox.  There’s a built in Date Picker, RepeatButton (you click it and a status bar gets progressively bigger), a 3-state button and a whole bunch of other shit no one (read: hardly anyone) ever uses, but one of the single most integral UI components IN HISTORY is currently absent.  What.  The.  Fuck.  Luckily writing your own combobox that works “well enough for now” is fairly easy, requiring maybe 5 lines of XAML and another 20 of C# code to do.  Microsoft promises (with sufficient fine print to allow them to back out of the promise at any given time) to have a combobox in by the time 2.0 is fully released and I, along with many others, are just crossing their fingers and praying it comes soon.  In the meantime, rolling your own that works well enough isn’t difficult at all.. writing your own that works exactly like you want it to on the other hand is not nearly as easy.  If you can’t wait for that, you can always pay $500-$1000 USD for something that you would normally never even consider paying money for.  Yeah, right.

 

Data Binding

The only other massive gripe (there’s a million minor ones) I currently have with Silverlight is the data binding.  .NET provided you with a whole host of different ways to bind controls to your data, but in general just about everyone (at least everyone I know) all chose the same route - DataSets.  You fill a DataSet with your column and row definitions and your data and just bind that to your control and everything worked automagically.  It was extremely useful and by a massive margin the single most used data binding feature ever implemented in .NET.  So, naturally, they chose to use this for Silverlight.. err, wait a minute, no they EXPLICITELY DENY YOU THE ABILITY TO DO THIS IN SILVERLIGHT.  THERE ARE NO FUCKING DATASETS!  The only way to bind data in Silverlight is to have an IEnumerable (say, a List<> object) of objects where each property of the object turns into a column.  So, say you have a class called Row with properties ‘Name’ ‘Date’ ‘Other’ and you have a list of 50 of those with different values for each property.  When you bind that to a DataGrid, you’ll get 3 columns and 50 rows with each column name equal to a property in that class.  Sound good?  Well, [un]lucky for us we will never ever ever ever ever ever ever ever have any idea what columns are going to be in our data sets until they’re there.  You can even add new columns in real-time.  We can’t possibly have a class that has each column enumerated as a property because WE DON’T KNOW WHAT THEY ARE TO MAKE THEM PROPERTIES.  With DataSets this was never a problem, since you never had to define ANYTHING at compile time.  While you could do it at compile time if you wanted to, in the background everything was still done at run-time.  Not so in Silverlight 2.0 Beta 1.  Currently the workaround I have for this is a massive ‘Row’ structure with hundreds of Col0-ColN properties and a constructor that takes one of our customly developed DataRow structures as a parameter and fills out the structure with one massive switch statement.  These DataRows are wrapped in a ‘DataSetViewable’ class that has a list of them and takes one of our custom DataSet structures as a parameter and fills everything in.  This structure can now be bound to Silverlight DataGrid controls directly and everything works mostly OK except for the fact that all the column titles are “Col0, Col1 .. ColN” with a few hundred of them not even having data.  So what we have to do is, after binding (if you do it before, everything crashes horribly) go back through the DataGrid’s column list and change the header name to what we actually want and mark all columns with no data as invisible.  Then everything looks and works fine.. from the user’s perspective at least.  Which is good enough for us for now until Microsoft get’s their shit together (they have promised, while simultaneously not gauranteeing, that this will be fixed before 2.0 is fully released).

Other than all that, which has made me seriously considering pistol whipping some of Microsoft’s developers a number of times, Silverlight has been a miracle worker for us.  Once you get around the retarded nuances and get your shit working, it actually stays working solidly and reliably - something I could never say for ASP.NET, AJAX, generic javascript, flash or just about any other web technology.  I should stress the fact that Silverlight 2.0 (which really should’ve been called 1.0) is still in beta stage at this point and everything me and my company have been encountering should be expected given the fact that we’re extremely early adopters of a technology that wasn’t even originally designed (as far as we can tell) to do what we’re doing with it.  But still - no ComboBox?  What the hell were you thinking?

 

That’s the end of my ranting for today.  My sister (who I haven’t seen for a few years now) is going to be in town all weekend, so I’m going to finish up here at work and go catch up with her.  Have a happy Memorial Day Weekend everyone! 

P.S.  I’ve updated my BRDF post with some more links to BRDF Databases.  I’m still missing a lot of my favorite databases, especially one which had around 40GB of data in it and was freely available to the public.  I’d really really love to find that.  If anyone comes across it, please send me a line.

Work in progress.. paused

Just a short update here..

 

Whelp, I just started work at a new job so I wouldn’t expect too many updates anytime soon (not that I post all that many updates to begin with!).  In case anyone’s keeping track the side projects I’ve been working on include a lot of graphics virtualization and streaming-world work, real-time BRDF [de]compression schemes, BRDF streaming schemes, a combination of the two into something i’ve been referring to as “Infinite Fidelilty Local Lighting”, some subdivision surface rendering stuff and a few miscelaneous other bits (the most notable being a new real-time texture magnification filter).  So far progress on all fronts has been very positive although I have, perhaps, overextended myself a bit.  Time permitting I’ll be working on all of this and I’ll provide updates as soon as information worthy of posting becomes available.  In the meantime, not much going on but work and enjoying the beach.

Neat Multiprocessing Links and Papers

Intel Threading Building Blocks -
“A primary benefit of TBB is that applications using the library automatically scale to utilize the available processing cores, with no changes to the source code or the executable program file. In other words, the same executable will create threads that utilize one core on a single core machine, two cores on a dual-core machine, four cores on a quad-core machine, etc. No recompilation of the application is required, because the library itself detects the hardware architecture and uses that information to determine how to break up the tasks for assignment to each core.” - From the Wiki Entry

---

The MIT ‘Adaptive Scheduling of Parallel Jobs’ Project -“In this project, we are investigating adaptive scheduling and resource allocation in the domain of dynamic multithreading. Most existing parallel programming systems are nonadaptive, where each job is assigned a fixed number of processors. This strategy may lead to a poor use of available resources. For example, if the job’s parallelism changes while the job is executing, or if the resources available in the system change, the job is still forced to run with the same number of processors as it was allotted when it started executing. A more attractive model would be an adaptive model, where processors allotted to a job change according to the job’s parallelism and the system environment.”

Some papers associated with this project:

Provably Efficient Two-level Adaptive Scheduling

Adaptive Task Scheduling with Parallelism Feedback

Adaptive Work Stealing with Parallelism Feedback

An Empirical Evaluation of Work Stealing with Parallelism Feedback

Dynamic Processor Allocation for Adaptively Parallel Work-Stealing Job

---

The Landscape of Parallel Computing Research: A View from Berkeley

---

If anyone has any others they’d like to mention, please feel free to leave a message in the comments section.

Bidirectional Reflectance Distribution Functions

Anyone with a keen eye that has used any of today’s Real-Time Rendered Graphics applications (read: played any games) has probably taken notice of the main topic of this entry – nothing lights like it is supposed to! We have all these bright, nice shiny bricks on our virtual buildings that are more accurately modeled with a strictly diffuse Lambert term (which we’ve been able to do real-time for close to 20 years) than with the Phong reflectance model that is used is most applications today. The Phong model could easily be tuned to model this effect very easily (just limit the specular and glossy components) but it appears that bricks laminated and coated with slime are more visually appealing to the graphics designers than something even reasonably plausible.

This is one of my biggest gripes with graphics today and has been for over 5 years. Not just with the over (and disgustingly inaccurate) dramatization of lighting effects that’s been going on but with the entire direction the lighting systems in modern real-time renderers have been heading in general. Most renderers take a single lighting model and apply it to ever surface in the world with some sort of per-surface tuning in the form of specular maps, gloss maps, etc. With a lot of fine-tuning by excellent artists, this can actually produce rather believable results. In some of the “better” rendering systems multiple lighting models are supported where you can draw some surfaces with the Phong lighting model, some with Ward, some with Lafortune, some with He-Torrance, etc. You’re still basically fighting a losing battle; however, as you’re trying to model what’s really a minimum of 4-dimensional function (the BRDF) with a series of 2D snap-shots and trying to figure out what “looks right”. The world would be a whole lot happier if developers would integrate some sort of actual BRDF data, measured from the real-world, into their pipeline to at least fit some of these analytical models to. There’s literally tons of free BRDF data available to anyone with an internet connection already. None of this have I seen used in any commercially available real-time rendering package.

Both of these solutions are analytical models used to approximate the true reflectance function of real surfaces – they’re guaranteed right off the bat to never be completely correct (and most of them aren’t even physically correct in theory). This is analogous to having a set of random points that you’d like to fit a curve to. The problem is we have to use a set function for all surfaces and fit drastically different (some almost completely incoherent) data to it – there’s bound to be a wide range of surfaces that fall by the wayside in this approximation.

A theoretically much more sound approach to all of this is to note that every surface in the real-world has its own distinct BRDF. Using one BRDF for all surfaces is just not going to cut it. Add a few more models to the mix and you’re definitely getting closer to what you want, but you’re still falling short. It’d be best if we had some sort of data driven approach where the lighting model wasn’t even a mathematical function as it is today and is simply some sort of “texture” (I use the term loosely here) that we just reference with the incident light vector, the outgoing eye vector and the wavelength of the light we’re trying to sample. We then set associate a BRDF for each texture/surface. In the lighting pass we simply reference the associate BRDF and boom – bricks look like bricks, plastic looks like plastic, wood looks like wood, etc no matter where you put the light and no matter what direction you view the surface from. This is essentially the holy grail of reflectance modeling (at least, after it has been extended to include sub-surface scattering and anisotropic reflections).

So where’s the problem? Remember a BRDF is, at a minimum, a 4-dimensional function – and we’re trying to tabulate all that data at a reasonable level of precision such that we don’t see any (much) banding or ringing. Clearly we now have a [massive] compression issue – we’d like to take all this data (that can easily be over 16MB per BRDF) and compress it in such a way that the memory print is acceptable (say 256-512kb), the core integrity of the data is preserved and we want to decompress this data on the fly extremely quickly. Luckily, there has been a good amount of research done in the academic community towards this end. Since the data we’re trying to compress is over the hemisphere of the point of incident, spherical compression models are an obvious first place to look. Among these we have (to name a couple) spherical harmonics and wavelets. Spherical harmonics have very promising results when we limit our scope to extremely low-frequency lighting, but as soon as you up the frequency of the lighting you start needing far too many coefficients to model the BRDF not to mention all the banding that occurs even with the copious amount of coefficients.

The solution, for the time being, seems to lye with wavelets or some form thereof. There has been a number of research papers published recently (and many not so recent) on using wavelets to compress BRDFs and subsequent decompression in real-time for use in real-time lighting. A couple of these papers manage to get quite impressive results too with BRDFs compressed to a maximum of around 256KB. The real-time rendering portions of their techniques leave a bit to desire, though – render rates are generally around 5-10 Hz with extremely simple scenes at low resolution on the fastest hardware around today. This actually boggles my mind a little, as I was able to make a few tech demos based on the techniques they describe in their papers (although using none of their code). I was able to achieve roughly equivalent results as far as quality goes, but the render rates were a full 10 orders of magnitude greater for far more complex scenes on far less sophisticated hardware. That’s the difference between a technique that never leaves the academic community and one that’s used as common practice in current and next generation games/real-time rendering.

I’m at a loss at what exactly was hampering their implementations so drastically, as all I did was to basically follow their instructions (granted their instructions were more on the conceptual level, leaving tons of room for varied implementations) and yet we came to two completely separate levels of performance in our final implementations. As soon as time (which is so incredibly short these days) allows, I plan to do a much more thorough investigation of what exactly is going on here.

Anyway, in summary – if we’re not there yet we’re damn close to being able to have completely data-driven, physically and aesthetically correct BRDFs defined on a per-surface basis. No more Phong shaders ran on every surface in the world, nor a list of hundreds of different light shaders that an artist has to weed through in order to find out which one works for the surface they’re trying to model. There will be one data-driven lighting function across all surfaces that allows all surfaces to be lit exactly as they should as defined by the BRDF associated with the surface.

Even if you don’t go this direction with your renderer – please, I beg of you, at least start using actual measured BRDF data to fit your lighting model to the surface you’re trying to represent. Using nothing but an artist’s eye to fit 4+ dimensional data to a single lighting model is never going to work out right. Fine tuning the data, after it’s been fit as closely as possible to the real-world data, is fine; but at least use some sound, actual data somewhere in your pipeline to make your fits. It won’t in anyway affect your rendering performance but it will make everything look a few orders of magnitude better.

Mathias Paulin’s Corner

Wavelet Encoding of BRDFs for Real-Time Rendering

Real-time Rendering with Wavelet-Compressed Multi-Dimensional Datasets on the GPU

Cornell Reflectance Data

MERL

CUReT

Experimental Analysis of BRDF Models (w/ BRDF Data)

[Coming: Lots more BRDF databases as soon as I dig out my favorites from my other computer]

My Nephew, Hayden

Baby Hayden
Hayden And His Dad
Go Red Sox!
The Patriot

So I finally made a blog..

…and I don’t have time to put anything on it

 ”Coming soon!”