Breaking News: Grepper is joining You.com. Read the official announcement!

pyspark multiple columns to one column json like structure with to_json example

Allie Ubisse answered on November 9, 2022 Popularity 3/10 Helpfulness 2/10

answer pyspark multiple columns to one column json like structure with to_json example

pyspark multiple columns to one column json like structure with to_json example

Comment

from pyspark.sql.functions import *

#sample data
df=spark.createDataFrame([('1234567','123 Main St','10SjtT','idk@gmail.com','ecom','direct')],['cust_id','address','store_id','email','sales_channel','category'])

df.select("cust_id","address",to_json(struct("store_id","category","sales_channel","email")).alias("metadata")).show(10,False)

#result
+-------+-----------+----------------------------------------------------------------------------------------+
|cust_id|address    |metadata                                                                                |
+-------+-----------+----------------------------------------------------------------------------------------+
|1234567|123 Main St|{"store_id":"10SjtT","category":"direct","sales_channel":"ecom","email":"idk@gmail.com"}|
+-------+-----------+----------------------------------------------------------------------------------------+

Popularity 3/10 Helpfulness 2/10 Language python

Source: stackoverflow.com

Tags: json multiple-columns pyspark python structure

Link to this answer
Share Copy Link

Contributed on Nov 09 2022

Allie Ubisse

0 Answers Avg Quality 2/10

pyspark multiple columns to one column json like structure with to_json example

Contents

More Related Answers

pyspark multiple columns to one column json like structure with to_json example

Grepper

Documentation

Social

Legal

Contact

Oops, You will need to install Grepper and log-in to perform this action.