Entries with tag "python"

Setting Up libRETS for Python

December 18, 2008

Tags: librets, mls, python, rets, web

As part of my business, I have taken on a good chunk of the local real estate market. For this non-technical market, it's surprising the amount of complexity involved with developing for this crowd. Real estate property listings have always been a very dynamic, yet necessary form of punishment for developers world-wide.

Some History

Most real estate offices(at least in North America) use a Multiple Listing Service(MLS) to advertise and keep track of their listings. This allows them to enter information, pricing, important dates, & pictures among other things about properties they list. Many of these (MLS) companies have a handful of extra tools available to the agents through their personal real estate CMS, for lack of a better universal name.

Traditionally, developers would use the listing feeds provided to them through this system. These are usually manifested as hideous HTML table layouts that most developers can spot a mile away. In an effort to make these listings visually bearable, developers like myself have been scraping these tables to extract the information so that it can be displayed with a little more style. This also offers the ability to perform more function with this information like sorting or keeping a history of listings. There are problems with this method however:

  • The layout is inconsitent — the table gets changed at seemingly random times. This will cause your scraping script to bug out and lead to an inbox full of agent hate mail. I'm not sure if this is because the company doesn't want you doing this as you can strip out all their branding, or weather the companies just don't like me. I lean towards the latter.
  • The opposite of RESTful — obviously the act of scraping a table is not the most desired way to obtain this information. There has to be a better way...

The Real Estate Transaction Standard

Apparently, enough people hated this as much as me and RETS was formed. RETS is a set of standards created to make obtaining this information more consistent throughout the different MLS associations. If you want to play with the big boys and have complete control over the information your presenting, this is the answer.

RETS data is obtained through an HTTP connection and has a standard set of fields. You can count on these to be the same with each different MLS association. This means developers can create everything from small scripts, to full web based applications for dealing with this data and not have to worry about changes to the layout. This leads to more stable solutions and mature distributable systems. There are packages to deal with these connections for many different platforms. The one I have chosen as a Django/Python developer is libRETS.

libRETS to the Rescue

libRETS is a C++ based library that offers the ability to use it through C++, Java, Perl, Php5, Python, Ruby, and the .NET languages. It is reliable, advanced, and by far the most mature option available for RETS connections.

It's available as a Windows binary installer or a *nix source package. Since I am using it on a linux production server and a Mac local development environment, I cannot offer any input on the Windows version. That said, the windows version is probably much easier to install, at least in my case.

I have a CentOS 5 production server, for those not familiar, CentOS has a great track record as being a reliable server enviroment. One of the main reasons is reliable package selection by default, a.k.a: usually old versions of everything. One of these packages was cURL, which libRETS uses heavily. There exists no modern version of cURL for CentOS 5. Some people have re-compiled Red Hat RPMs of a newer cURL in this case, I decided just to use an older version of libRETS. Time is money.

Installing libRETS

Download and extract RETS to a suitable location:

# wget http://www.crt.realtors.org/projects/rets/librets/files/librets-1.2.5.tar.gz
# tar zxvf librets-1.2.5.tar.gz
Currently, the latest version is 1.2.5, others are available here.

Install libRETS:

# cd librets-1.2.5
# ./configure
# make
# sudo make install
I'm not getting into troubleshooting problems here, this implies you have all the dependencies required to do this. Google is your friend.

Then you need to install the Python bindings. RETS uses SWIG for the bindings, if you do not have it installed, you must do this first. You should be able to use your package manager of choice to install SWIG. For example on CentOS, I used yum. Once you have SWIG installed, do this:

# cd projects/swig
# make python
# cd python
# python setup.py install

This should put the libRETS library in your PYTHONPATH. Test it by doing this:

# python
> import librets
If it doesn't complain, you are in business. If it does, then either it did not install correctly or it is not on your PYTHONPATH. Again, Google is your friend.

Using libRETS

Before doing any monkeying with libRETS, it's a good idea to test with a basic python script first. The docs here have a great line-by-line walk through on getting your first script working. I would suggest you follow that and make sure things are working first if your not familiar with libRETS.

Soon I will be posting the second part of this tutorial, Using libRETS with Django.