Question

Let's say I have the following input in apache pig:

(123, ( (1, 2), (3, 4) ) )
(666, ( (8, 9), (10, 11), (3, 4) ) )

and I want to convert these 2 rows into the following 7 rows:

(123, (1, 2) )
(123, (3, 4) )
(666, (8, 9) )
(666, (10, 11) )
(666, (3, 4) )

i.e. this is sorta 'doing the opposite of a GROUP'. Is this possible in pig latin?

Was it helpful?

Solution

Take a look at FLATTEN. It does what you probably need.

However, using your notation above, it looks like the list of tuples is a tuple. This should be a bag for this to work properly.

Instead of:

(123, ( (1, 2), (3, 4) ) )
(666, ( (8, 9), (10, 11), (3, 4) ) )

You should be representing your data as:

(123, { (1, 2), (3, 4) } )
(666, { (8, 9), (10, 11), (3, 4) } )

Then, once it is this form, you can do:

O = FOREACH grouped GENERATE $0, FLATTEN($1);
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top