Looking for late 90's game

When I was a kid I played a certain game that I'm now eager to find to revive old memories.

It involved controlling a "masculine" male character that was set in the middle of the screen if I remember correctly. You moved around with him and shot at enemies, and throughout the game you acquire new weapons, one that I specifically remember was almost like toothpaste, you blasted this big stream of toothpaste resembling substance at your enemies. I think you also had dynamites that you could throw, and whenever you threw one he would say "fire in the hole!"

The first stage was a type of jungle. The only other stage I remember was some kind of candy land, with walls made up of big pieces of hard candy. And I think maybe there was a type of castle stage in the clouds.

The animation style was cartoonish and the tracks were sort of like diablo or diablo 2, so as you move around it sort of feels like 3D. I have no idea what year it could be from, but I'm thinking 95-99.

I also have no idea what the title is, but "joe" comes into mind, maybe just because i remember the main character to be a masculine looking cartoon figure. I wasn't able to find it by searching however.

If anyone knows what game this is please help me out! Thanks