Interesting point Irvin, about the word "Thereby". Nowadays, it seems that it has become increasingly replaced with the word "Therefore". They are oftentimes used interchangeably, however, there does appear to be a difference between their definitions.

"Therefore" means "For that reason", whereas "Thereby" means "By that means" or "Because of that".

After contemplating many various examples, I cannot think of a specific case in which the use of one of the adverbs is favored over the other. In every example, it appears that both words can be successfully interchanged. I would love to see an example that proves otherwise. More often than not, I'm fairly sure that it comes down to the individual writer to determine which of these words, along with hence, thus, etc., sound or look more aesthetically pleasing.

Ex: After weeks of intense debates, the delegates finally came to an accord and signed the Nuclear Disarmament Treaty, which thereby calls for the immediate and complete abolition of nuclear arms.

(I thought I had just thought of one, but I guess "therefore" also works here. However, I feel that "thereby" looks more formal, which appeals to the context, and sounds slightly better than "therefore". Although, grammatically speaking - both are explicitly correct.

And just to be a certified grammar nerd, in reference to your example, "I'm broke, hence I will walk rather than taking the bus."; "taking" should actually be replaced with "take". ;)

Furthermore, for the last part. Wouldn't it sound even better if that sentence became "This thing is a balloon, thereby it is made of rubber"? Basically remove the "and" while adding the "it". Moreover, replacing "thereby" with "therefore" in this example still works perfectly. And for some reason, I think "therefore" sounds better in this case. Any thoughts?