How to evaluate the integral
$$
I_p=int_0^infty fraclog xx^p (x-1), dx,?$$

Attempt:

My first idea was to find the area of convergence, which gives me $p in (0,1)$. Now let's consider the function $f(z) = fraclog^2 zz^p (z-1)$. Then we can use Cauchy theorem and residuals.

But I'm stuck here. Because there is no term to describe $I_1$. Any ideas?

We can take from here the following representation of the digamma function:
$$psi(s+1)=-gamma+int_0^1 frac1-x^s1-xdxRightarrow psi_1(s)=int_0^1 fracx^s-1ln xx-1dx$$
Now we are just one step away to solve it by splitting the integral into two pieces because we have:
$$int_1^infty fracln xx^p(x-1)dxoversetlarge x=frac1t=int_0^1 fract^p-1ln tt-1dt$$

$$int_0^infty fracln xx^p(x-1)dx=int_0^1 fracx^-pln xx-1dx+int_0^1fracx^p-1ln xx-1dx=psi_1(1-p)+psi_1(p)=boxedfracpi^2sin^2 (pi p)$$
Above follows by the the trigamma's reflection formula.

Well use $$Re s>0implies int_0^inftyfracue^-su1-e^-udu=sum_nge 0frac1(n+s)^2.$$With the substitution $u=ln x$ followed by the identity $int_Bbb Rg(u)du=int_0^infty [g(u)+g(-u)]du$, $$I_1=int_0^inftyfracu (e^-pu+e^-(1-p)u)1-e^-udu=sum_nge 0left(frac1(n+1-p)^2+frac1(n+p)^2right).$$
This is evaluated here as $pi^2csc^2pi p.$

The OP stated at the end of the posted question

Now let's consider the function $f(z) = fraclog^2 zz^p (z-1)$. Then we can use Cauchy theorem and residuals.

Rather than analyze a contour integral of $f(z)=fraclog^2 zz^p (z-1)$, we analyze the integral

$$I(p)=oint_C fraclog(z)z^p(z-1),dz$$

where $C$ is the classical keyhole contour where the branch cut for $log(z)$ is take along the positive real axis and $arg(z)in [0,2pi)$. Inasmuch as $fraclog(z)z^p(z-1)$ is analytic in and on $C$, Cauchy's Integral Theorem guarantees that

$$beginalign
0&=oint_C fraclog(z)z^p(z-1),dz\\
&=int_epsilon^R fraclog(x)x^p(x-1),dx+underbraceint_0^2pi fraclog(Re^iphi)R^pe^ipphi(Re^iphi-1),iRe^iphi,dphi_to 0,,textas,,Rto infty+int_R^1+nufraclog(x)+i2pix^pe^i2pi p(x-1),dx\\
&+underbraceint_2pi^pi fraclog(1+nu e^iphi)(1+nu e^iphi)^pnu e^iphi,inu e^iphi,dphi_to -ipi (i2pi )e^-i2pi p,,textas,,nuto0+int_1-nu^epsilon fraclog(x)+i2pix^pe^i2pi p(x-1),dx+underbraceint_2pi^0 fraclog(epsilon e^iphi)(epsilon e^iphi)^p(epsilon e^iphi-1),iepsilon e^iphi,dphi_to 0,,textas,,epsilonto 0tag1
endalign$$

Letting $epsilonto 0^+$, $Rto infty$, and $nu to 0^+$ in $(1)$, we obtain

$$(1-e^-i2pi p)int_0^infty fraclog(x)x^p(x-1),dx=i2pi e^-i2pi p,,textPVleft(int_0^infty frac1x^p(x-1),dxright)+(ipi)left(i2pi e^-i2pi p right)tag2$$

where $textPVleft(int_0^infty frac1x^p(x-1),dx+ipiright)$ denotes the Cauchy Principal Value (CPV) integral.

We can evaluate the CVP integral as follows.

$$beginalign
textPVint_0^infty frac1x^p(x-1),dx&=lim_nuto 0^+left(int_0^1-nufrac1x^p(x-1),dx+int_1+nu^inftyfrac1x^p(x-1),dxright)\\
&=lim_nuto 0^+left(-sum_n=0^inftyint_0^1-nux^n-p,dx+sum_n=0^inftyint_1+nu^infty x^-n-p-1,dxright)\\
&=sum_n=0^infty left(frac1n+p-frac1n-p+1right)tag3
endalign$$

The series on the right-hand side of $(3)$ is the partial fraction representation of $pi cot(pi p)$ for $pin (0,1)$ (SEE HERE).

Substituting $(3)$ into $(2)$, dividing both sides by $(1-e^-i2pi p)$, we find

$$int_0^infty fraclog(x)x^p(x-1),dx=fracpi^2sin^2(pi p)$$

Too Long for comment

If $p≥2$ were some natural number, we could write the indefinite integral as,
$$
I = log x left[ log left( 1-frac 1x right) + sum_r=2^p frac 1(r-1)x^r-1 right] - text Li _2 left( frac 1x right) + sum_r=2^p frac 1(r-1)^2x^r-1 + C
$$
However, I don't know how this can be related to your problem, as you say $0<p<1$ which seems absurd.