Scrapping websites

i have a project in mind and zero programming experience. I’d like to scrape websites for real estate listings, collecting data points fro the listing, and providing a daily output of what is new. Where would I want to start to develop project to accomplish this? Just looking for a very high level outline so I can find a starting point.

Thx in advance, John.

If you have 0 programming experience Id consider looking to learn how to program as a good first step.

I personally would recommend learning python. Why Python? Python is known for being very approachable and great to learn as a starting language due to its clear syntax and flexibility

The other key reason(s) I would recommend python is because it has a huge amount of libraries which would help you on this sort of project.

  • libraries for web scraping? Yup Beautiful Soup
  • daily outputs? (I assume in a text file?) Yup (cron job, or website, or something)
  • data analysis Yup, python is known for being great for data science (if you cared about having python do more than give you data, it can be easily used to get info from the data)

You obviously can do this in other languages (like Nodejs, or Java) but in terms of getting started, getting going fast and staying focused on you current requirements (zero programming experience, webscraping, collecting data, and using said data) python is one of the best choices, and is a very popular choice too.

Goodluck :smiley:

Brad, thanks. This is what I was hoping to receive as feedback. It’s a helpful outline and nudge in a particular direction. I am interested to see if others concur.:fu:

Randel, I appreciate the feedback. It sounds like you’re an expert on MLS listings and developing listing websites for brokers. However, before you imply that my intended gathering of information is an illegitimate effort, perhaps you could ask me to clarify my intended use? I have no interest in MLS properties or the brokering of said properties or facilitating any broker or agent in their sales activities. There are many types of real estate and parties with an interest at any given time. Don’t be so presumptuous and cast my interest and intentions as anything other than legitimate in a public forum without ascertaining contextual facts.