The Problems With UK Property Portal Feeds, And How They Can Grow Up
Update – 10th April 2014
It’s been over a year since I wrote this post and now it seems that, at last, we are seeing the beginnings of a move to real-time updates. Rightmove have recently introduced the ability to send property data in realtime. As far as we’re aware, these are the only ones to offer this at this time. No doubt however, others will follow in their footsteps soon.
Having worked in the property industry for many years now I can confidently say that I’ve seen property portals evolve over time, both from an industry point of view and a technological one. I’ve seen hundreds of property portals start up to then only never be heard of again a few weeks later, and I’ve seen other companies merge to provide bigger and better sites.
To kick things off, here’s a statistic that I want you to consider whilst reading this post:
What I want to highlight is the fact that these property portals are key in the selling and letting of homes.
Before I continue it’s worth noting that, although this post is primarily focused on the exchange of data between the data providers and the portals from a backend point of view, it should also be of interest to users of the portals (ie. house hunters) and the estate agents who’s data is getting sent thoughout the World Wide Web.
If you are reading this as an estate agent, and you use some kind of software or automated method to send your property data to the portals, answer me this one question:
Can you explain, in detail, how your properties (ie. the core of your business) get sent to the portals, and who has access to them?
It’s also worth noting at this point that the majority of the points I’m about to explain are not applicable to all portals and are targeted toward the property portal industry in general.
1. Real Time Data
I’ll start with my biggest gripe first in that the property information sent to the portals is not processed in realtime. The world is a fast moving place and, with sites like Facebook and Twitter, users now expect data the second that it is available. The difference between property data not being available to view for a couple of hours could be the difference between a potential buyer seeing a property and not seeing a property.
What’s the solution? For the portals to provide an API so data can be sent, received and processed right away. To put it into context, this could reduce the time between a property being made available and it being displayed on the property portal from hours, to seconds.
Another advantage of real time feeds would be the process of correcting errors. As it stands, if an error is made in the property details and is sent to the portal it can take a few hours for a property to be sent again and the incorrect information put right. With a live feed the data would be updated immediately.
No, I haven’t made a typo. I’ve put security at joint first in my list for two reasons; 1) To make people aware of just how insecure these portals are and 2) To make people aware of just how insecure these portals are. Again, not a typo. Allow me to explain the problems of the current method:
Data is sent via FTP – FTP stands for ‘File Transfer Protocol’ and allows data (in this case properties) to be moved betweeen servers. The problem is that all data is sent unencrypted meaning, if someone really wanted to and knew what they were doing, they could get a hold of an agents entire available stock, reports, as well as any enquiries made against these properties from the beginning of time. The link below explains more about FTP and it’s insecurities:
Usernames and passwords sent via email – To send data via the FTP method mentioned above you must have a username and password. The majority of portals will happily send this information around via email at the drop of a hat without thinking twice. You might be thinking “well, what’s the problem with that?”. I’ll leave you to read this small piece which I hope will make it clear:
Some might call me too security concious, after all, this isn’t credit card information that we’re sending around the web. That’s true, but it is an estate agent’s livelihood. They’ve worked hard to win and instruct these properties and they’ve spent money on advertising to generate interest. Would they really want someone outside of their orgainsation seeing all of their enquiries? It’s also worth mentioning that more information is actually sent to portals in the feeds than what they actually display on their website.
2. Lack of reporting when something goes wrong
Nothing is perfect in this world, and the same applies to portal feeds. Errors are made, data isn’t always in the format expected, and sometimes a little investigation is needed to know why a property isn’t appearing on the portals (or if was even sent at all). Of the hundreds of property portals I’ve seen, there is only a handful that provide some kind of reporting that allows the agent or data provider to immediately see the information that was received, processed and/or rejected when a feed is sent.
Without reporting it means emails and phone calls back and forth, and ultimately extra time to fix what is normally a simply mistake.
3. Lack of two way information
As with the above, there are only a handful of property portals I know of that provide two way information and make it available to the agent and/or software that initially sent the feed. What information do I mean? I mean enquiries and website stats (for example how many times has a property been viewed).
Again, this information would ideally be made available in real time and transferred securely.
4. Flat files
My fifth and final point is to do with the structure of the file that is sent. For those of you that haven’t been involved in the setup of a feed before, the industry standard has come to be a format known as a BLM file, originally introduced by Rightmove. Much like a text file or CSV (Comma Separated Values), the file is two-dimensional meaning all property information needs to go on it’s own row. If extra information is added a later date the row gets longer and it becomes very difficult to link multiple rows together and create relationships.
Other disadvantages of this format mean that data always has be sent in the same order and if a certain field doesn’t exist for a property then the field has to be sent anyway. For example, if an agent specialises in sales and lettings, the rent frequency field will be sent for every sales propert,y even though it is completely irrelevant.
For now I see the BLM format sticking around, however it would be nice to see the industry move towards a more efficient, quicker, expandable and futureproof format soon.
I am in no way saying that property portals do not work. I use them on a regular basis, my friends and family use them and they’ve become a part of our everyday lives. What I am saying however is that there is definitely room for improvement.
Remember the initial diagram at the beginning of this post? With property portals being such a huge part of the public’s house hunting, I think it’s important that these feeds and the data sent are handled as best as can be to give the users the most up-to-date and accurate information, in what will probably be one of the biggest decisions of their lives.