3)\(...=\left[\frac{\left(\sqrt{x}+\sqrt{y}\right)\left(1+\sqrt{xy}\right)+\left(\sqrt{x}-\sqrt{y}\right)\left(1-\sqrt{xy}\right)}{\left(1-\sqrt{xy}\right)\left(1+\sqrt{xy}\right)}\right].\frac{1-xy}{x+xy}\)

= \(\frac{\sqrt{x}+x\sqrt{y}+\sqrt{y}+y\sqrt{x}+\sqrt{x}-x\sqrt{y}-\sqrt{y}+y\sqrt{x}}{1-xy}.\frac{1-xy}{x\left(1+y\right)}\)= \(\frac{2\sqrt{x}+2y\sqrt{x}}{x\left(1+y\right)}=\frac{2\sqrt{x}\left(1+y\right)}{x\left(1+y\right)}=\frac{2}{\sqrt{x}}\)

Áp dụng BĐT Cô si ta có:

\(x+y\ge2\sqrt{xy}=2\cdot\frac{1}{\sqrt{z}};y+z\ge2\sqrt{yz}=2\cdot\frac{1}{\sqrt{x}};z+x\ge2\sqrt{xz}=2\cdot\frac{1}{\sqrt{y}}.\)( vì xyz=1)

=> P\(\ge\)\(\frac{2x\sqrt{x}}{y\sqrt{y}+2z\sqrt{z}}\)+ \(\frac{2y\sqrt{y}}{z\sqrt{z}+2x\sqrt{x}}+\frac{2z\sqrt{z}}{x\sqrt{x}+2y\sqrt{y}}\)

Đặt \(\hept{\begin{cases}a=y\sqrt{y}+2z\sqrt{z}\\b=z\sqrt{z}+2x\sqrt{x}\\c=x\sqrt{x}+2y\sqrt{y}\end{cases}\left(a;b;c\ge0\right)}\)<=> \(\hept{\begin{cases}4a+b=2c+9z\sqrt{z}\\4b+c=2a+9x\sqrt{x}\\4c+a=2b+9y\sqrt{y}\end{cases}}\)

<=> \(\hept{\begin{cases}z\sqrt{z}=\frac{4a+b-2c}{9}\\x\sqrt{x}=\frac{4b+c-2a}{9}\\y\sqrt{y}=\frac{4c+a-2b}{9}\end{cases}}\)

Do đó:

P \(\ge\)\(\frac{2}{9}\cdot\left(\frac{4a+b-2c}{c}+\frac{4b+c-2a}{a}+\frac{4c+a-2b}{b}\right)\)

<=> P \(\ge\)\(\frac{2}{9}\left(4\left(\frac{a}{c}+\frac{b}{a}+\frac{c}{b}\right)+\left(\frac{b}{c}+\frac{c}{a}+\frac{a}{b}\right)-6\right)\)

<=> P \(\ge\frac{2}{9}\cdot\left(4\cdot3\cdot\sqrt[3]{\frac{a}{c}\cdot\frac{b}{a}\cdot\frac{c}{b}}+3\cdot\sqrt[3]{\frac{b}{c}\cdot\frac{c}{a}\cdot\frac{a}{b}}-6\right)\)( Áp dụng BĐT Cô si cho 3 số ko âm)

<=> P \(\ge\frac{2}{9}\left(12+3-6\right)=2\)( đpcm)

Dấu = khi x=y=z=1.

c/

\(\left(x-4\right)P+y^2+2xy+1+\left|2x+3y+1\right|=0\)

\(\Leftrightarrow\frac{\left(x-4\right)\left(x^2-1\right)}{x-4}+y^2+2xy+1+\left|2x+3y+1\right|=0\)

\(\Leftrightarrow x^2+y^2+2xy+\left|2x+3y+1\right|=0\)

\(\Leftrightarrow\left(x+y\right)^2+\left|2x+3y+1\right|=0\)

Do \(\left\{{}\begin{matrix}\left(x+y\right)^2\ge0\\\left|2x+3y+1\right|\ge0\end{matrix}\right.\) \(\forall x;y\)

\(\Leftrightarrow\left\{{}\begin{matrix}x+y=0\\2x+3y+1=0\end{matrix}\right.\) \(\Rightarrow\left\{{}\begin{matrix}x=1\\y=-1\end{matrix}\right.\)

ĐKXĐ: \(x\ge0;x\ne4\)

\(P=\left(\frac{\sqrt{x}+2}{\sqrt{x}+3}+\frac{x^2-x+3}{\left(\sqrt{x}+3\right)\left(\sqrt{x}-2\right)}\right):\left(\frac{\sqrt{x}}{\sqrt{x}+2}+\frac{\sqrt{x}+4}{\left(\sqrt{x}+2\right)\left(\sqrt{x}+3\right)}\right)\)

\(P=\left(\frac{x-4+x^2-x+3}{\left(\sqrt{x}+3\right)\left(\sqrt{x}-2\right)}\right):\left(\frac{x+3\sqrt{x}+\sqrt{x}+4}{\left(\sqrt{x}+3\right)\left(\sqrt{x}+2\right)}\right)\)

\(P=\left(\frac{x^2-1}{\left(\sqrt{x}+3\right)\left(\sqrt{x}-2\right)}\right)\left(\frac{\left(\sqrt{x}+3\right)\left(\sqrt{x}+2\right)}{\left(\sqrt{x}+2\right)^2}\right)\)

\(P=\frac{x^2-1}{\left(\sqrt{x}+3\right)\left(\sqrt{x}-2\right)}.\left(\frac{\sqrt{x}+3}{\sqrt{x}+2}\right)\)

\(P=\frac{x^2-1}{x-4}\)

b/ Để \(P\ge0\Leftrightarrow\frac{x^2-1}{x-4}\ge0\Leftrightarrow\left[{}\begin{matrix}\left\{{}\begin{matrix}x^2-1\ge0\\x-4>0\end{matrix}\right.\\\left\{{}\begin{matrix}x^2-1\le0\\x-4< 0\end{matrix}\right.\end{matrix}\right.\) \(\Leftrightarrow\left[{}\begin{matrix}x>4\\-1\le x\le1\end{matrix}\right.\)

Kết hợp với ĐKXĐ \(x\ge0\), \(\Leftrightarrow\left[{}\begin{matrix}x>4\\0\le x\le1\end{matrix}\right.\)