Yes, $\operatorname{card}(\mathcal I(\mathcal L))\le\operatorname{card}(X)$. For each $x\in\bigcup\mathcal L$, let $M_x=\bigcap\{A\in\mathcal L:x\in A\}$, the least element of $\mathcal L$ containing $x$. Clearly $M_x$ is join-irreducible. These are the only join-irreducible elements: if $A\in\mathcal L$ then $A=\bigcup\{M_x:x\in A\}$, so $A$ is not join-irreducible unless $A=M_x$ for some $x\in A$.
More generally, suppose $\mathcal L$ is a sub-lattice of a finite distributive lattice $\mathcal L'$, and let $1_\mathcal L$ be the greatest element of $\mathcal L$. For each $a\in\mathcal I(\mathcal L')$ with $a\le1_\mathcal L$, define $\hat a=\bigwedge\{x\in\mathcal L:a\le x\}$; then $a\le\hat a\in\mathcal I(\mathcal L)$.
To see that $\hat a$ is join-irreducible in $\mathcal L$, suppose $\hat a=b\vee c$ for some $b,c\in\mathcal L$. By distributivity, $a=ab\vee ac$; since $a$ is join-irreducible in $\mathcal L'$, either $a=ab$ or else
$a=ac$. Say $a=ab$; then $b\in\{x\in\mathcal L:a\le x\}$, so $\hat a\le b\le\hat a$, so $\hat a=b$.
For any $b\in\mathcal L$ we have $b=\bigvee\{a:a\in\mathcal I(\mathcal L'),\ a\le b\}=\bigvee\{\hat a:a\in\mathcal I(\mathcal L'),\ a\le b\}$; if $b$ is join-irreducible in $\mathcal L$, then $b=\hat a$ for some $a\in\mathcal I(\mathcal L')$. Hence
$$\mathcal I(\mathcal L)=\{\hat a:a\in\mathcal I(\mathcal L'),\ a\le1_\mathcal L\}$$
and so
$$\operatorname{card}(\mathcal I(\mathcal L))\le\operatorname{card}(\{a\in\mathcal I(\mathcal L')):a\le1_\mathcal L\})\le\operatorname{card}(\mathcal I(\mathcal L')).$$