I have a dataset as below
| col1 | extension_col1 |
|---|---|
| 2345 | 2246 |
| 2246 | 2134 |
| 2134 | 2091 |
| 2091 | Null |
| 1234 | 1111 |
| 1111 | Null |
I need to find the number of extensions available for each record in col1. the final result as below
| col1 | extension_col1 | No_Of_Extensions |
|---|---|---|
| 2345 | 2246 | 3 |
| 2246 | 2134 | 2 |
| 2134 | 2091 | 1 |
| 2091 | Null | 0 |
| 1234 | 1111 | 1 |
| 1111 | Null | 0 |
value 2345 extends as 2345>2246>2134>2091>null and hence it has 3 extension relations excluding null.
How to get the 3rd column(No_Of_Extensions) using spark sql/scala.?
