Categories

Efficient algorithm to obtain near optimal policies for an MDP

Given a discrete, finite Markov Decision Process (MDP) with its usual parameters $(S, A, T, R, \gamma)$, it is possible to obtain the optimal policy $\pi^{*}$ and the optimal value function $V^{*}$ through one of many planning methods (policy iteration, value iteration or solving a linear program). I am interested in obtaining a random near-optimal […]

Categories

Efficient algorithm to obtain near optimal policies for an MDP

Given a discrete, finite Markov Decision Process (MDP) with its usual parameters $(S, A, T, R, \gamma)$, it is possible to obtain the optimal policy $\pi^{*}$ and the optimal value function $V^{*}$ through one of many planning methods (policy iteration, value iteration or solving a linear program). I am interested in obtaining a random near-optimal […]

Categories

Find the minimizer $u$ of a functional whose derivate contains both $u$ and the sign of $u$

Let $N,h,\gamma,\bar x$ be constants and $x_i\in[-1,1],\forall i=\overline{1,N}$. Consider the following problem $$\min_u J(u^n) \quad\text{subject to}\quad x_i^{n+1} = x_i^n+hu^n$$ where $$J(u^n) = \frac{1}{2N}\sum_{i=1}^N |x_i^{n+1}-\bar x|^2+h\gamma|u^n|$$ Consider the problem on a single time interval $[t^n,t^{n+1}]$ and plug the expression for $x_i^{n+1}$ inside $J$ $$J(u^n) = \frac{1}{2N}\sum_i|x_i^n+hu^n-\bar x|^2+h\gamma|u^n|$$ Derive wrt $u$ […]

Categories

How to find the minimizer u of a functional whose derivate contains both u and the sign of u?

Let $N,h,\gamma,\bar x$ be constants and $x_i\in[-1,1],\forall i=\overline{1,N}$. Consider the following problem $$\min_u J(u^n) \quad\text{subject to}\quad x_i^{n+1} = x_i^n+hu^n$$ where $$J(u^n) = \frac{1}{2N}\sum_{i=1}^N |x_i^{n+1}(u)-\bar x|^2+h\gamma|u^n|$$ Consider the problem on a single time interval $[t^n,t^{n+1}]$ and plug the expression for $x_i^{n+1}$ inside $J$ $$J(u^n) = \frac{1}{2N}\sum_i|x_i^n+hu^n-\bar x|^2+h\gamma|u^n|$$ Derive wrt $u$ […]

Categories

The maximum for $xy \sin \alpha + yz \sin \beta +zx \sin \gamma$.

Question: Deduce the maximum of $xy \sin \alpha + yz \sin \beta +zx \sin \gamma$ if $x,y,z$ are real numbers that satisfy $x^2+3y^2+4z^2=6$ with $0<\alpha,\beta,\gamma<\pi$ such that $\alpha+\beta+\gamma=2\pi$. Currently, I am not very sure how to approach the problem. I had an idea to consider the area of a triangle made of 3 smaller triangles […]

Categories

Do Carmo Riemannian Geometry Exercise 4.5

Let $\gamma:[0,1]\rightarrow M$ be a geodesic and let $X$ be a vector field on $M$ such that $X(\gamma(0))=0$. Show that $$\nabla_{\gamma’}(R(\gamma’,X)\gamma’)(0) = (R(\gamma’, X’)\gamma’)(0)$$ where $X’ = \frac{DX}{dt}$. The solution is simple by using derivative of tensors:for all $Z$ on $M$ and at $t=0$, we have \begin{align} 0 &= (\nabla_{\gamma’}R)(\gamma’, X, \gamma’,Z) \\ &=\frac{d}{dt}\langle R(\gamma’,X)\gamma’, […]

Categories

How to prove this ? I have tried to use $F$ but it doesn’t fulfill the conditions. Thank you.

Let $X$ be a topological space, $I=[0,1]$ be a subspace of usual space $\mathbb{R}$ and $\dot{I}=\{0,1\}$. Let $F:I \times I \rightarrow X$ be continuous. Let $\alpha, \beta, \gamma, \delta$ be paths in $X$. If $F(t,0)=\alpha(t), F(t,1)=\beta(t), F(0,t)=\gamma(t), F(1,t)=\delta(t)$, then prove that $\alpha \simeq \gamma \star \beta \star \delta^{-1}$ rel $\dot{I}$. Note : if $f$ and […]

Categories

Hints for proving that the collection of all set functions whose domain is the set $X$ and whose range is a subset or equal to the set $Y$ is a set.

I am trying to figure out how to formally write down a set that contains as its elements sets of ordered pairs with a very specific property (i.e. I want $\Omega=\{A,B,C,…\}$ where $A=\{(\alpha,\beta),(\alpha’,\beta’),…\}, B=\{(\gamma,\zeta),(\gamma’,\zeta’),…\}$ etc). The specific property of each set belonging to $\Omega$ can be stated as follows: Let $D \in \Omega$ be any […]

Categories

How to prove this ? I have tried to use $F$ but it doesn’t fulfill the conditions. Thank you.

Let $X$ be a topological space, $I=[0,1]$ be a subspace of usual space $\mathbb{R}$ and $\dot{I}=\{0,1\}$. Let $F:I \times I \rightarrow X$ be continuous. Let $\alpha, \beta, \gamma, \delta$ be paths in $X$. If $F(t,0)=\alpha(t), F(t,1)=\beta(t), F(0,t)=\gamma(t), F(1,t)=\delta(t)$, then prove that $\alpha \simeq \gamma \star \beta \star \delta^{-1}$ rel $\dot{I}$. Note : if $f$ and […]

Categories

Confluent Heun function with parameter $\alpha = 0$

I am solving a problem that led me to the confluent Heun equation, which is: $$\dfrac{d^2H(z)}{dz^2}+\bigg(\alpha + \dfrac{\beta +1}{z} + \dfrac{\gamma +1}{z-1} \bigg)\dfrac{dH(z)}{dz} + \bigg(\dfrac{\mu}{z}+\dfrac{\nu}{z-1}\bigg)H(z)=0,$$ of which a power series solution can be found around the origin: $$H(z) = HeunC(\alpha,\beta,\gamma,\delta,\eta;z) = \sum_{n=0}^{\infty}v_n(\alpha,\beta,\gamma,\delta,\eta)z^n.$$ $HeunC(\alpha,\beta,\gamma,\delta,\eta;z)$ is known as confluent Heun function. Here, $z$ is a variable defined […]