Software Engineering Daily-logo

Software Engineering Daily

Business


Episodes

Materialize: Streaming SQL on Timely Data with Arjun Narayan and Frank McSherry

2/19/2020
Distributed stream processing frameworks are used to rapidly ingest and aggregate large volumes of incoming data. These frameworks often require the application developer to write imperative logic describing how that data should be processed. For example, a high volume of clickstream data that is getting buffered to Kafka needs to have a stream processing system The post Materialize: Streaming SQL on Timely Data with Arjun Narayan and Frank McSherry appeared first on Software Engineering...

Duration:01:11:09

Go Networking with Sneha Inguva

2/18/2020
A cloud provider gives developers access to virtualized server infrastructure. When a developer rents this infrastructure via an API call, a virtual server is instantiated on physical machines. That virtual server needs to be made addressable through the allocation of an IP address to make it reachable from the open Internet. When the virtual server The post Go Networking with Sneha Inguva appeared first on Software Engineering Daily.

Duration:00:58:18

Great Expectations: Data Pipeline Testing with Abe Gong

2/17/2020
A data pipeline is a series of steps that takes large data sets and creates usable results from them. At the beginning of a data pipeline, a data set might be pulled from a database, a distributed file system, or a Kafka topic. Throughout a data pipeline, different data sets are joined, filtered, and statistically The post Great Expectations: Data Pipeline Testing with Abe Gong appeared first on Software Engineering Daily.

Duration:01:08:49

Data Warehouse ETL with Matthew Scullion

2/14/2020
A data warehouse provides low latency access to large volumes of data. A data warehouse is a crucial piece of infrastructure for a large company, because it can be used to answer complex questions involving a large number of data points. But a data warehouse usually cannot hold all of a company’s data at any The post Data Warehouse ETL with Matthew Scullion appeared first on Software Engineering Daily.

Duration:00:57:53

Anyscale with Ion Stoica

2/13/2020
Machine learning applications are widely deployed across the software industry. Most of these applications used supervised learning, a process in which labeled data sets are used to find correlations between the labels and the trends in that underlying data. But supervised learning is only one application of machine learning. Another broad set of machine learning The post Anyscale with Ion Stoica appeared first on Software Engineering Daily.

Duration:00:56:51

Flink and BEAM Stream Processing with Maximilian Michels

2/12/2020
Distributed stream processing systems are used to read large volumes of data and perform operations across those data streams. These stream processing systems often build off of the MapReduce algorithm for collecting and aggregating large volumes of data, but instead of processing a calculation over a single large batch of data, they process data on The post Flink and BEAM Stream Processing with Maximilian Michels appeared first on Software Engineering Daily.

Duration:00:51:14

Druid Analytics with Jad Naous

2/11/2020
Large companies generate large volumes of data. This data gets dumped into a data lake for long-term storage, then pulled into memory for processing and analysis. Once it is in memory, it is often read into a dashboard, which presents a human with a visualization of the data. The end-user who is consuming this data The post Druid Analytics with Jad Naous appeared first on Software Engineering Daily.

Duration:00:56:50

The Data Exchange with Ben Lorica

2/10/2020
Data infrastructure has been transformed over the last fifteen years. The open source Hadoop project led to the creation of multiple companies based around commercializing the MapReduce algorithm and Hadoop distributed file system. Cheap cloud storage popularized the usage of data lakes. Cheap cloud servers led to wide experimentation for data tools. Apache Spark emerged The post The Data Exchange with Ben Lorica appeared first on Software Engineering Daily.

Duration:01:08:35

Presto with Justin Borgman

2/7/2020
A data platform contains all of the data that a company has accumulated over the years. Across a data platform, there is a multitude of data sources: databases, a data lake, data warehouses, a distributed queue like Kafka, and external data sources like Salesforce and Zendesk. A user of the data platform often has a The post Presto with Justin Borgman appeared first on Software Engineering Daily.

Duration:01:16:18

Nubank Data Engineering with Sujith Nair

2/6/2020
Nubank is a popular bank that is based in Brazil. Nubank has more than 20 million customers, and has accumulated a high volume of data over the six years since it was started. Mobile computing and cloud computing have given rise to “challenger banks” that operate more like software companies. When a software company reaches The post Nubank Data Engineering with Sujith Nair appeared first on Software Engineering Daily.

Duration:01:04:55

Changelog Podcasting with Adam Stacoviak and Jerod Santo

2/5/2020
The Changelog is a podcast about the world of open source. As open source has become closely tied with the entire software development lifecycle, The Changelog has expanded its coverage to the broader software industry. Since starting the podcast ten years ago, Adam Stacoviak and Jerod Santo have become full-time podcasters, and they have started The post Changelog Podcasting with Adam Stacoviak and Jerod Santo appeared first on Software Engineering Daily.

Duration:01:15:44

Rive: Animation Tooling with Guido and Luigi Rosso

2/4/2020
Animations can be used to create games, app tutorials, and user interface components. Animations can be seen in messaging apps, where animated reactions can convey rich feelings over a text interface. Loading screens can become less boring through animation, and voice assistant products can feel more alive through animation. But we still don’t see much The post Rive: Animation Tooling with Guido and Luigi Rosso appeared first on Software Engineering Daily.

Duration:01:22:09

John Deere: Farm Software with Ryan Bergman

2/3/2020
Robotics has changed modern agriculture. Autonomous systems are powering the tractors, cotton pickers, and corn cutters that yield plants at industrial scale. John Deere is a company that has been making farm equipment for 183 years. Over that period, the planting and harvesting process has become increasingly mechanized, and John Deere has been at the The post John Deere: Farm Software with Ryan Bergman appeared first on Software Engineering Daily.

Duration:01:02:34

Venture Stories with Erik Torenberg

1/31/2020
Venture capital investing requires an understanding of market dynamics, technology, and finance. There is also an element of human nature. Consumer trends can make or break the viability of a new product. And early stage venture investing is always a bet on a small team or individual founder. Early stage investments are usually into companies The post Venture Stories with Erik Torenberg appeared first on Software Engineering Daily.

Duration:00:56:13

Alpaca: Stock Trading API with Yoshi Yokokawa

1/30/2020
Stock trading takes place across a variety of software platforms. Etrade and Schwab have allowed individual traders to buy securities for decades. Robinhood built a business around a similar model, but also removed the commission. Wealthfront and Betterment provide “roboadvisor” services that abstract away the underlying securities and focus on managing a risk profile. Each The post Alpaca: Stock Trading API with Yoshi Yokokawa appeared first on Software Engineering Daily.

Duration:01:10:58

Cloud Log Analysis with Jack Naglieri

1/29/2020
Large software companies have lots of users, and the activity from those users results in high volumes of traffic. These companies also have a large surface area across the enterprise. There are hundreds of services and databases that are fulfilling user requests. As these requests enter the infrastructure of the enterprise, the requests travel through The post Cloud Log Analysis with Jack Naglieri appeared first on Software Engineering Daily.

Duration:01:07:54

Replicated Software Delivery with Grant Miller and Marc Campbell

1/28/2020
Distributed systems are required to run most modern enterprise software. Application services need multiple instances for scalability and failover. Large databases are sharded onto multiple nodes. Logging services, streaming frameworks, and continuous integration tools all require the orchestration of more than one server. Deploying a distributed system has historically been difficult because the nodes of The post Replicated Software Delivery with Grant Miller and Marc Campbell appeared...

Duration:01:08:51

Mattermost with Ian Tien

1/27/2020
Chat systems have been a part of software development for decades. Older systems like Pidgin and Yammer were surpassed by newer systems like HipChat. And when Slack was created, it quickly became a part of most software companies. But Slack does not fulfill the needs of every company. Mattermost is an open-source chat system. Mattermost The post Mattermost with Ian Tien appeared first on Software Engineering Daily.

Duration:00:51:27

GitLab Strategy with Sid Sibrandij

1/24/2020
The word “DevOps” has a different definition depending on who you ask. For some people, it is about the process of managing and releasing code. It can involve container management and server orchestration. It can involve infrastructure-as-code, and safer configuration management. In addition to a set of technologies, DevOps can be seen as a management The post GitLab Strategy with Sid Sibrandij appeared first on Software Engineering Daily.

Duration:01:05:17

Lyft Kubernetes with Vicki Cheung

1/23/2020
The ridesharing infrastructure of Lyft has a high volume of traffic that is mostly handled by servers on AWS. When Vicki Cheung joined Lyft in 2018, the company was managing containers with an internally built container scheduler. One of her primary goals at the company was to move Lyft to Kubernetes. In today’s episode, Vicki The post Lyft Kubernetes with Vicki Cheung appeared first on Software Engineering Daily.

Duration:00:49:13