Consider a $\Bbb C$-vector space $V$ ($\Bbb C$ just because it's easy to visualize). Now consider the set of all linear functions $f:V\to \Bbb C$. That set is called the dual space to $V$ and is denoted $V^*$. You can easily confirm (by going through each of the axioms) that $V^*$ is also a vector space. In fact if $V$ is finite dimensional then $V^*$ has exactly the same dimension as a $\Bbb C$-vector space. Here addition and scalar multiplication work exactly as they do for any functions: let $v\in V$, $f,g\in V^*$, and $k\in \Bbb C$ then $$(f+g)(v) = f(v) + g(v) \\ (kf)(v) = k(f(v))$$
So because any vector space has an associated dual space, we could call some particular element of either space a "vector". But that would be ambiguous. So people have given names to vectors in each vector space. Vectors in $V$ are sometimes called "contravariant vectors", "kets", or even just regular "vectors" while vectors in $V^*$ are often called "dual vectors", "covectors", "covariant vectors", or "bras".
And yes, we physicists will usually write kets with the notation $\lvert v\rangle$ and bras with the notation $\langle v\rvert$. We don't normally explain why we're always allowed to associate some unique bra $\langle v\rvert$ with any given ket $\lvert v\rangle$ like this, we just take the Riesz Representation Theorem for granted (at least my professors always seem to).
Going a little further we could consider a differentiable manifold $M$. Attached to any point $p$ in $M$ is a "tangent space" and a "cotangent space". We consider the tangent space $T_pM$ to be the space of "contravariant" vectors or kets. These kets are the vectors you get by taking directional derivatives at the point $p\in M$ (actually we usually consider the directional derivative operators themselves to be the vectors).
Then the cotangent space is the space dual to it so we denote it $T^*_pM$ and call its elements "covariant" vectors or bras (or any of those other names). These bras are just the linear functions $\alpha: T_pM \to \Bbb C$ (assuming $\Bbb C$ is still the base field).