Using Pulley to Pull, Sync, and Debug Listing Data! Yassss!

Start by logging in to kickstart:

(Can't login? Talk to Kevin or Brian!)

Change to the /scripts/ directory:

Now you can run some commands to debug and pull listing data. Here is the general syntax:

Here is how to view all the available commands:

You can then view the args (arguments) for each command such as this example for debug:

That first section before ".sh" has a few options as well:

./foreground.sh will run the command and display the results of the pull as it runs in your terminal
./background.sh will run the command and dump out a hash that you can then use in papertrail to see the pull
./debug.sh will run the command and display ALL the information that happens when you run the pull, this is useful for debugging RETS errors.

Next we have the section before ".py":

./runner.py is used for the majority of things you run.
./kickstart.py is used for syncing a board, this is commonly used for pulling sold data.
./supervisor.py is used for the processes that are running constantly to keep data up to date.

You can use"--help" to find a list of commands, here's a list:

debug
debug_photo
find_and_refresh
find_and_regeocode
full_photo
full_prop
geocode
incr_photo
incr_prop
missing_images
missing_properties
refresh_known_properties
refresh_property
retry_stuck_images
unremove_properties
verify_properties

Gross, what are all those things?

Well, they are all used for specific tasks, for now we will focus on debug, and we can revisit these other options at another time.

ANOTHER TIME IS NOW, 3 YEARS LATER.

debug = one off check a single listing's data
debug_photo = one off check a single listing's photos
find_and_refresh = refresh a specific set of listings based on a mongo query
find_and_regeocode = doesn't work, but I gather if it did, it would refresh a specific set of listings' geocoding based on a mongo query
full_photo = pull all photos for board.
full_prop = pull all listings for board.
geocode = run geocode for entire board. (only runs for listings that don't already have geocoding fields in mongo)
geolookup_properties Used only with chicago
incr_photo = used mostly for testing a board, pulls a chunk of photos.
incr_prop = used mostly for testing a board, pulls a chunk of listings.
missing_images = pulls missing images. (properties that don't have photos in mongo)
missing_properties = pulls missing properties. (properties that don't exist in mongo)
refresh_known_properties = refreshes the listings that exist in mongo.
refresh_property = refreshes an individual listing.
retry_stuck_images = haven't used this one in a loooong time, I imagine it retries stuck photos?
unremove_properties = sometimes a board will randomly disappear a pile of listings, this tries to refresh all listings marked as removed in mongo.
verify_properties = this checks if what we have in mongo exists in data feed and removes it if it doesnt.

 

Here's an example debug command:

In this example, we are debugging MLS# "123152" on board "132". After you run this, you will see an output of the fields for this listing.

Hopefully at this point, you have an example of whatever it is you are debugging this listing for. You can now use the Find command to search for the information you are seeking.

But like, what if, you know, I debug it, and no data is returned?

Well, in the majority of these cases this is due to this listing not being enabled for IDX display. There could also be a typo in your command, or maybe it's on a different MLS board the client is on, but isn't attached to their site.

Well that's great, but what do I do about it?

Easy peasy. First, it never hurts to do a quick google search to see if that MLS# is displaying on other peoples sites on the internet somewhere. If you can find it on a bunch of IDX sites, then maybe there's an issue on our end.

Guy, you're still not telling me what to tell the client.

Geez, calm down. Fine. Here's what I would tell the client:

Hi Bob! I was unable to locate MLS#123456 in our data feed. Can you confirm this listing is approved for IDX display?

Cool, that wasn't so hard.

Right? So now if the client says "Yeah it is approved" blergh, it's time to email the MLS board about this listing. You can review other emails that are sent out of our realgeeks.help@gmail.com email address to get a good example of what to send. You want to make sure to include the MLS#, and provide our RETS credentials (WITH NO PASSWORD).

Cool. Great Doc Rite?

Damn straight.