8 votes

Topic deleted by author

3 comments

  1. [3]
    Akir
    Link
    This is the standout passage that explains everything to me: Hadoop got big because it was du jour. It's just like it's broader categorical cousin, NoSQL. Both were new sexy technologies that were...

    This is the standout passage that explains everything to me:

    Companies were reminded that they had already invested untold billions in relational database and data warehouse technologies that actually worked pretty well. And everyone already knew SQL.

    Hadoop got big because it was du jour. It's just like it's broader categorical cousin, NoSQL. Both were new sexy technologies that were implemented in new projects because they were the new hotness, not because they were the best tools for the job.

    It's a real shame that developers don't seem to understand the value of a good RDBMS. It's like a mass scale "not invented here" syndrome. And as a result there is a new database system that gains huge support at least once a year. And with so many smaller projects I see that use ORMs, I wonder if programmers are simply afraid of SQL.

    5 votes
    1. [2]
      archevel
      Link Parent
      I don't believe this is true and the article seems to contradict it. Developers have realized that RDBMSs are great at representing relational data. That doesn't mean all data should be stored in...

      It's a real shame that developers don't seem to understand the value of a good RDBMS.

      I don't believe this is true and the article seems to contradict it. Developers have realized that RDBMSs are great at representing relational data. That doesn't mean all data should be stored in a RDBMS, but there is a lot of information in the world that is suitably modelled in that way.

      For processing realtime streams of data I'm not entirely convinced a RDBMS is a great choice, but for analysing and querying semi-stable data it is awesome. I want to push a bit for using regular command line tools for simple analysis though. You can get quite far processing data with just unique, sort, cut, xargs and awk, especially if you are just processing less then a terrabyte if data.

      As a side note, wouldn't be nice if SQL had switched the order of FROM and SELECT clause. I often know where the data I am interested in is, but not necessarily exactly the column names I want. I almost always end up writing SELECT * FROM then filling out the from clause and then going back to pick our the right columns/aggregations... Anyway, just a pet peeve with SQL.

      2 votes
      1. Akir
        Link Parent
        Perhaps I should be more specific in what I mean. I am certainly not saying that RDBMSes are the best for any data storage situation. The real problem that developers have is that they aren't...

        Perhaps I should be more specific in what I mean. I am certainly not saying that RDBMSes are the best for any data storage situation. The real problem that developers have is that they aren't great at choosing the best technology to solve their problems. I will bet that there are a ton of internal projects implementing GraphQL even though they don't have the problems that GraphQL was invented to solve simply because it's newer and part of the zeitgeist. The same could have been said about the adoption of Hadoop and NoSQL databases. When those poor choices are made, those projects tend to just disappear unless they have enough commercial support to rewrite to use a better option.

        2 votes