Breaking News: Grepper is joining You.com. Read the official announcement!

to_json pyspark

Add Answer

Tanmay Dabre answered on October 6, 2022 Popularity 4/10 Helpfulness 1/10

answer to_json pyspark

related to_json pyspark

related pyspark transform df to json

related to_json pyspark

to_json pyspark

Comment

>>> newDf1 = df.withColumn("jsonCol", to_json(struct([df[x] for x in df.columns])))
>>> newDf1.show(10,False)
+-----+----+---+------+--------------------------------------------------------+
|id   |name|age|salary|jsonCol                                                 |
+-----+----+---+------+--------------------------------------------------------+
|10001|alex|30 |75000 |{"id":"10001","name":"alex","age":"30","salary":"75000"}|
|10002|bob |31 |80000 |{"id":"10002","name":"bob","age":"31","salary":"80000"} |
|10003|deb |31 |80000 |{"id":"10003","name":"deb","age":"31","salary":"80000"} |
|10004|john|33 |85000 |{"id":"10004","name":"john","age":"33","salary":"85000"}|
|10005|sam |30 |75000 |{"id":"10005","name":"sam","age":"30","salary":"75000"} |
+-----+----+---+------+--------------------------------------------------------+

Popularity 4/10 Helpfulness 1/10 Language python

Source: stackoverflow.com

Tags: pyspark python

Link to this answer
Share Copy Link

Contributed on Oct 06 2022

Tanmay Dabre

0 Answers Avg Quality 2/10

Closely Related Answers

to_json pyspark

Comment

>>> newDf1 = df.withColumn("jsonCol", to_json(struct([df[x] for x in df.columns])))
>>> newDf1.show(10,False)
+-----+----+---+------+--------------------------------------------------------+
|id   |name|age|salary|jsonCol                                                 |
+-----+----+---+------+--------------------------------------------------------+
|10001|alex|30 |75000 |{"id":"10001","name":"alex","age":"30","salary":"75000"}|
|10002|bob |31 |80000 |{"id":"10002","name":"bob","age":"31","salary":"80000"} |
|10003|deb |31 |80000 |{"id":"10003","name":"deb","age":"31","salary":"80000"} |
|10004|john|33 |85000 |{"id":"10004","name":"john","age":"33","salary":"85000"}|
|10005|sam |30 |75000 |{"id":"10005","name":"sam","age":"30","salary":"75000"} |
+-----+----+---+------+--------------------------------------------------------+

Popularity 6/10 Helpfulness 5/10 Language python

Source: stackoverflow.com

Tags: python pyspark

Link to this answer
Share Copy Link

Contributed on Oct 06 2022

Tanmay Dabre

0 Answers Avg Quality 2/10

pyspark transform df to json

Comment

results = df.toJSON().map(json.loads).collect()

Popularity 6/10 Helpfulness 5/10 Language javascript

Source: stackoverflow.com

Tags: to-json pyspark transform javascript transf

Link to this answer
Share Copy Link

Contributed on Feb 15 2023

Caio Henrique

0 Answers Avg Quality 2/10

to_json pyspark

Comment

>>> newDf2 = df.withColumn("jsonCol", to_json(struct([ when(col(x)!="  ",df[x]).otherwise(None) for x in df.columns])))
>>> newDf2.show(10,False)
+-----+----+---+------+---------------------------------------------------------+
|id   |name|age|salary|jsonCol                                                  |
+-----+----+---+------+---------------------------------------------------------+
|10001|alex|30 |75000 |{"col1":"10001","col2":"alex","col3":"30","col4":"75000"}|
|10002|bob |31 |80000 |{"col1":"10002","col2":"bob","col3":"31","col4":"80000"} |
|10003|deb |31 |80000 |{"col1":"10003","col2":"deb","col3":"31","col4":"80000"} |
|10004|john|33 |85000 |{"col1":"10004","col2":"john","col3":"33","col4":"85000"}|
|10005|sam |30 |75000 |{"col1":"10005","col2":"sam","col3":"30","col4":"75000"} |
+-----+----+---+------+---------------------------------------------------------+

Popularity 5/10 Helpfulness 3/10 Language javascript

Source: stackoverflow.com

Tags: pyspark javascript

Link to this answer
Share Copy Link

Contributed on Oct 06 2022

Tanmay Dabre

0 Answers Avg Quality 2/10

to_json pyspark

Comment

>>> newDf2 = df.withColumn("jsonCol", to_json(struct([ when(col(x)!="  ",df[x]).otherwise(None) for x in df.columns])))
>>> newDf2.show(10,False)
+-----+----+---+------+---------------------------------------------------------+
|id   |name|age|salary|jsonCol                                                  |
+-----+----+---+------+---------------------------------------------------------+
|10001|alex|30 |75000 |{"col1":"10001","col2":"alex","col3":"30","col4":"75000"}|
|10002|bob |31 |80000 |{"col1":"10002","col2":"bob","col3":"31","col4":"80000"} |
|10003|deb |31 |80000 |{"col1":"10003","col2":"deb","col3":"31","col4":"80000"} |
|10004|john|33 |85000 |{"col1":"10004","col2":"john","col3":"33","col4":"85000"}|
|10005|sam |30 |75000 |{"col1":"10005","col2":"sam","col3":"30","col4":"75000"} |
+-----+----+---+------+---------------------------------------------------------+

Popularity 5/10 Helpfulness 2/10 Language javascript

Source: stackoverflow.com

Tags: pyspark javascript

Link to this answer
Share Copy Link

Contributed on Oct 06 2022

Tanmay Dabre

0 Answers Avg Quality 2/10

to_json pyspark

Comment

>>> newDf2 = df.withColumn("jsonCol", to_json(struct([ when(col(x)!="  ",df[x]).otherwise(None) for x in df.columns])))
>>> newDf2.show(10,False)
+-----+----+---+------+---------------------------------------------------------+
|id   |name|age|salary|jsonCol                                                  |
+-----+----+---+------+---------------------------------------------------------+
|10001|alex|30 |75000 |{"col1":"10001","col2":"alex","col3":"30","col4":"75000"}|
|10002|bob |31 |80000 |{"col1":"10002","col2":"bob","col3":"31","col4":"80000"} |
|10003|deb |31 |80000 |{"col1":"10003","col2":"deb","col3":"31","col4":"80000"} |
|10004|john|33 |85000 |{"col1":"10004","col2":"john","col3":"33","col4":"85000"}|
|10005|sam |30 |75000 |{"col1":"10005","col2":"sam","col3":"30","col4":"75000"} |
+-----+----+---+------+---------------------------------------------------------+

Popularity 5/10 Helpfulness 1/10 Language javascript

Source: stackoverflow.com

Tags: javascript pyspark

Link to this answer
Share Copy Link

Contributed on Oct 06 2022

Tanmay Dabre

0 Answers Avg Quality 2/10

to_json pyspark

Comment

>>> newDf2 = df.withColumn("jsonCol", to_json(struct([ when(col(x)!="  ",df[x]).otherwise(None) for x in df.columns])))
>>> newDf2.show(10,False)
+-----+----+---+------+---------------------------------------------------------+
|id   |name|age|salary|jsonCol                                                  |
+-----+----+---+------+---------------------------------------------------------+
|10001|alex|30 |75000 |{"col1":"10001","col2":"alex","col3":"30","col4":"75000"}|
|10002|bob |31 |80000 |{"col1":"10002","col2":"bob","col3":"31","col4":"80000"} |
|10003|deb |31 |80000 |{"col1":"10003","col2":"deb","col3":"31","col4":"80000"} |
|10004|john|33 |85000 |{"col1":"10004","col2":"john","col3":"33","col4":"85000"}|
|10005|sam |30 |75000 |{"col1":"10005","col2":"sam","col3":"30","col4":"75000"} |
+-----+----+---+------+---------------------------------------------------------+

Popularity 4/10 Helpfulness 1/10 Language javascript

Source: stackoverflow.com

Tags: javascript pyspark

Link to this answer
Share Copy Link

Contributed on Oct 06 2022

Tanmay Dabre

0 Answers Avg Quality 2/10

to_json pyspark

Contents

More Related Answers

to_json pyspark

Closely Related Answers

to_json pyspark

pyspark transform df to json

to_json pyspark

to_json pyspark

to_json pyspark

to_json pyspark

Grepper

Documentation

Social

Legal

Contact

Oops, You will need to install Grepper and log-in to perform this action.