Ask HN: Who is hiring? (January 2023)
ycombinator.com - get the latest breaking news, showbiz & celebrity photos, sport news & rumours, viral videos and top stories from ycombinator.com Daily Mail and Mail on Sunday newspapers.
Ask HN: Who is hiring? (January 2022)
ycombinator.com - get the latest breaking news, showbiz & celebrity photos, sport news & rumours, viral videos and top stories from ycombinator.com Daily Mail and Mail on Sunday newspapers.
Meanwhile, AWS s data lake – Elastic Map Reduce – can handle SQL queries via SQL Workbench or Presto SQL. Azure supports SQL queries in its data lakes (HD Insight or Azure Databricks) while GCP uses a combination of Bigtable, Dataflow and Bigquery.
But these implementations are not able to handle the number of SQL queries supported by traditional data warehouses, some of which can scale to thousands of concurrent users. Latency and concurrency an issue
Pal told the Gartner Data & Analytics Summit: Data lakes are actually not being used for BI workloads, especially in large organizations that need high concurrency, as well as low latency. The SQL engines that have been developed on the data lakes have never really been able to keep up with the concurrency and latency requirements.