You’ve got a bunch of CSV files and you’ve heard of Parquet. How do you convert them for Azure Synapse Analytics? Patrick shows you how using pySpark.

pyspark DataFrame
https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/dataframe.html

pyspark.sql.DataFrameReader.load
https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrameReader.load.html

pyspark.sql.DataFrameWriter
https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrameWriter.html

📢 Become a member: https://guyinacu.be/membership

*******************

Want to take your Power BI skills to the next level? We have training courses available to help you with your journey.

🎓 Guy in a Cube courses: https://guyinacu.be/courses

*******************
LET’S CONNECT!
*******************

— http://twitter.com/guyinacube
— http://twitter.com/awsaxton
— http://twitter.com/patrickdba
— http://www.facebook.com/guyinacube
— https://www.instagram.com/guyinacube/
— https://guyinacube.com

***Gear***
🛠 Continue reading “Convert CSV to Parquet using pySpark in Azure Synapse Analytics” →