@98Problems I know what you mean about needing to compare different locations. We had to do the same thing. We used www.USA.com and Bestplaces Compare Cost of Living.
For your website idea, you will need to have a database or two with large amounts of data. You can use Access, MySQL, PostgreSQL, or MongoDB. The database commands are pretty close to each other. However, if you want your website to scale up, it would be best to go with MySQL, PostreSQL or MongoDB.
For pulling from other sites, you would need to have their permission, which might cost good amount of money except for Data.gov.
Also, trying to maintain static websites would be pain unless you use something like Jenkins as a static site generator.
1 Like
deciphering …
Yes, Lindsay, you’ll definitely need a database, preferably PostgreSQL and not MongoDB or MySQL.
There’s a thing called relational schema normalization, or database normalization, I suggest to get into it - it allows to reduce the amount of redundancy and improve data integrity.
99% startups what I did and what I’ve seen before had a lack of it, and the cost of developing a business logic alongside a denormalized data set grew exponentially.
The trick with PostgreSQL is that you can use some advanced techniques and indexing solutions to work with massive amounts of data that will grow over time. So you’ll don’t have to worry about how much terabytes of stats you’ll be importing. Some of the data should be stored on SSD drives, and most of the imported stats should be receding on plain old RAID arrays.
Feel free to poke me on skype: void.nugget
I’ll gladly help you with database schema designs… materialized views, pl/sql functions, triggers, trigger constraints, all the good stuff people are usually omitting with an excuse of overcomplification. I usually tend to recognize it as a laziness and rigidity.
1 Like