Bing's dogbone space is a non-manifold $X$ such that $X\times\mathbb R\cong\mathbb R^4$. It is constructed as a quotient of $\mathbb R^3$ which is the identity outside of a ball. Hence we can do the construction inside a ball, to get a modified dogbone space $W$ with $\partial W=S^2$. Then I think that $W\times\mathbb R\cong D^3\times \mathbb R,$ which has boundary $S^2\times\mathbb R$. The basic idea as I understand it is that the nested tangle of genus 2 handlebodies unknots itself in $4$ dimensions, and this doesn't appear to use anything outside of a ball. However I haven't ever gone through the proof in detail, so I might be missing something.
If you take $Y=[0,1]$ I would guess that $X$ is probably a manifold with boundary. I recall hearing in a lecture many years ago that if $X\times S^1$ is a manifold, then so is $X$, which strikes me as a very similar problem.