Your solution is not totally a hack. Actually is pointing in the right direction. The iup.fill{} element can be used to do that, it is a void element that does exactly that. But since it expands only in the direction of the box, a solution will look like this:
tab1 = iup.hbox {
iup.button { title = "A button" },
iup.fill { },
iup.vbox{iup.fill { }}
;
tabtitle = "Tab1"
}