Transformation matrix in alternate basis

In my assignment I have a vector space of polynomials with the degree at most 3 described by this linear tranformation.

I have to write the transformation matrix in the basis {1,x,x^2,x^3} for R3[x] and standard basis R^3.

If we write the polynomial of degree 3 in vector form we get [c1,c2,c3,c4]^T. The linear transformation is than of the form:

fi([c1,c2,c3,c4]^T) = [c1 - c2 + c3 + c4, c1, c1 + c2 + c3 + c4]

If I understand correctly than to get the transformation matrix for the basis {1,x,x^2,x^3} I apply the transformation to the standard basis of R^4: [1,0,0,0],[0,1,0,0],[0,0,1,0],[0,0,0,1].So I get the vectors, which, joined together, will form the new transformation matrix = A. I get (by rows): [{1,-1,1,-1},{1,0,0,0},{1,1,1,1}].

To get the transformation matrix of R^3 I use the formula D = PAP^-1, where P will be the identity matrix of R^3 and A is the matrix which I got before.

-

Your $A$ is correct, but the "$D=PAP^{-1}$" part is problematic. The matrix $A$ is 3-by-4. It is not a square matrix and the product $PAP^{-1}$ does not make any sense. Also, if I understand correctly, the assignment problem only asks you to find the transformation matrix with respect to the cannonical bases of $\mathbb{R}_3[x]$ and $\mathbb{R}^3$. So you have already solved the problem after you determined $A$. There is no need to perform any change of bases.