Suppose there is a fiber bundle $E$. The base space is $M$ so that $\pi:E\rightarrow M$ is the projection. By the definition, the bundle has a typical fiber $F$ such that the local trivialization over one open subset $U_i$ of $M$ is defined in this way $\phi_i:U_i\times F\rightarrow \pi^{-1}(U_i)$. So we can find out that no matter which open set $U_i$ we choose, we always choose the same typical fiber $F$. I guess this is because we want to have a smooth structure. If at different points of $M$, we choose different fibers, for example, at point $p\in M$ we attach a 3 dimensional vector space, while at point $q\in M$ we attach a 2 dimensional vector space and so on, then we are not going to have a smooth structure.
So my question is: can we attach different fibers to different points of a base manifold? If we insist to do so, is this structure still smooth? Is it still called a fiber bundle?