Loss function
Mean Square Error
Loss=21i∑(yi−aiL)2 ∂aiL∂Loss=−(yi−aiL) Binary Cross-Entropy
Loss=i∑[−yilogaiL−(1−yi)log(1−aiL)] ∂aiL∂Loss=aiL(1−aiL)−(yi−aiL) Activation
logistic
ail=σ(zil)=logistic(zil)=1+e−zil1 ∂zil∂σ(zil)=(1+e−zil)2e−zil=ail(1−ail)