The beer-ware license

Бахти якия лиценз!

"THE BEER-WARE LICENSE" (Revision 42):

As long as you retain this notice you can do whatever you want with this stuff. If we meet some day, and you think this stuff is worth it, you can buy us a beer in return.

This project is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
The beer-ware license

MD/MHD като първо име

От доста време се чудя защо някой азиатци във името си имат Md или Mhd. Дори за известно време си мислех, че е нещо като Dr (доктор) или Eng (инжинер). После си мислех, че е MD (Medical Doctor) при това много сериозно.

Оказа се обаче, че това е късата форма на Muhammad или Mohammad. Може също така и да се види и като Mohd.

MD/MHD като първо име

Safari ITP 2.1 DEMO Showing cookies expiring in 7 days

A few days ago, WebKit announced that the new Safari versions will include an updated version of Intelligent Tracking Prevention 2.1. Here’s a link to the announcement:
https://webkit.org/blog/8613/intelligent-tracking-prevention-2-1/
Now, with a few tests, I’ll show you how this affects you directly. Below you will also find an example of how to bypass the protection.

First, we will start with an usual test of how a cookie is created locally through an API document.cookie. The file that we will use can be found here:
https://www.mobilio.bg/demo_cookies.html
The methodology is as follows. Upon loading the website, you can see that no cookies appear. After that we create them by pressing the buttons 1-3 and again displaying the cookies. These cookies have an expiry date somewhere around 2021, 2022 and 2023. For convenience, the developer tools have been enabled as to display these same cookies. The images shown are before and after. In the time between the tests the cookies jar has been cleaned as to avoid any interference in the results.

We start with Firefox:

As you can see, the cookies have been created and their expiry dates are several years ahead in the future.

Next comes Chrome:

Similarly, there isn’t any change here. Cookies have several years expiry.

What follows is a desktop Safari (macOS):

And this is where we see ITP in action. Cookies are still the same, but their lives (their expiration is) are limited to a week

Our last test is the Safari (iOS) for mobile. This is where things get a little more complicated. Before the test, let’s first purge the cookie jar. To do that we go to Settings, Safari, Advanced and do the following:

:

Now that all cookies are purged, we can experiment.

After that, we go back to Settings, Safari, Advanced, Website Data and see that we got some data.

The problem is that we can’t see what these data are. That’s why we need to connect the iOS device with a cable to a macOS and connect to the device from a desktop Safari. Here’s a screenshot:

As you can see, these cookies are capped to a seven day expiry, just like they are for a desktop Safari.

Let’s continue testing this time with the cookies sent by the server itself. For this purpose we will use the following file:
https://www.mobilio.bg/demo_server_cookies.php
If your are interested in its code here it is:
https://www.mobilio.bg/demo_server_cookies.txt
The methodology here is the following. We load the file, we make a screenshot. We reload the file and make another screenshot.

Firefox:

Here, as you can see the life of the cookies is 10 years.

Chrome:

10 years again.

Desktop Safari (macOS):

Here too we see our cookie with 10 years of expiry.

Mobile Safari (iOS):

Here again we need to connect the iOS with macOS in order to see properly the data.

I intentionally left the old cookies here together with the new ones so that you can see how 3 of them have a life of one week and one (the new one) of them expires in 2029.

Why will this little change play a big role in the future?

Over the years, many tracking, advertising and analytics systems began to systematically abuse online tracing. This is the reason our browsers are stuck from piles of useless cookies that we, as users, do not need at all, they just don’t serve us. The only people for whom cookies carry some information are the owners of the above-mentioned networks.
In fact, just loading an ad code leads to the recording of many many cookies from other advertising networks because of their resale free advertising slots.
Social sharing buttons are another example. When I’m logged on to a social network I have an authentication cookie on my device. And when I visit a site with social sharing buttons in it, based on these cookies, the social network knows which sites I have been visiting.
There are other examples, but let’s not dig into too much technical stuff.

But why will we still be affected anyway?

After the hasty joy of less tracking comes the sad news. Alas, we, as webmasters, will also be affected just because of the use of Analytics products.

The main problem is how these products use cookies. We will take the popular Google Analytics as an example. The use of their cookies is well described here:
https://developers.google.com/analytics/devguides/collection/analyticsjs/cookie-usage
For example, we will use 2 scenarios:

  1. This scenario is entering the website from somewhere and performing some action. The simplest example is visiting a site for locksmiths repairs, plumbers or even HVAC repairs. In this case, the user performs usually one action which is most often to find their way on the website and see whether they can use the services for their particiular district. After a phone contact follows. This same visitor may or may not return within the next couple of days. In this case, perhaps 80-90% of the site visits are from new users.
  2. This scenario is a bit more complicated and involves several visits of the user. A simple example is the news publisher or an e-commerce site. In this case the user visits the site from some source – search, ads, social media or directly. Then they continue visiting the website – directly, via search, remarketing, social media, mail marketing, etc. Due to the nature of these visits, it is important to measure the first visit and the user’s entire path across the website in all his returning visits. If this is e-commerce, the duration can be a week, but very often the journey itself can last for weeks or even months. We may also have a one-time purchase or multiple purchases over time.

Traditionally Google Analytics keeps the cookies for 2 years from the moment of the first visit. This is more than enough to measure user activity. This also makes it possible to accordingly analyze the new and the returning users.

The answer to these two scenarios is relatively easy.

In the first case, we have a low interactivity on the website and therefore the the loss of cookies will not affect the measurement of the users. Additionally, the website target audience are their new clients from the website. Having used the service, visitors usually do not return to the website. In fact, these websites are not so pesky for the reporting of the user activity. So if you have a website falling into this category you have nothing to worry about.

In the second case, however, we have a website with high interactivity. Here the loss of cookies is already fatal for the website. From now on, the website will not be able to monitor and measure effectively user activity. It it these websites that are the real danger, entering your personal space. Typically, there is one or more ad networks, one or several analytics scripts, and several tracking scripts. At the same time, they have implemented integration with at least one social network. If you have a website falling in this category, you have all the sufficent grounds to worry about this change.

Is there a fix for this?

Of course!
Here is an example of a fix (together with the code needed for it):
https://www.mobilio.bg/demo_server_cookies_ext.php
https://www.mobilio.bg/demo_server_cookies_ext.txt
After the first cookie has been generated, if the user uses a login button, the server takes the cookie, changes it and extends its expiry to 2 years.
Here’s an example of desktop Safari too:

And now, let’s make a pseudo login which will extend the life of our cookie to 10 years:

As you can see, you can extend the life of an existing cookie. But, unfortunately, this will require the calling of some code on a server. And many of the tracking, advertisements and analytics products and services use serverless solutions to deliver information to their end customers. Now these services will have to change the way they deliver files to end-users if they want to offer their users the right reporting for Safari users beyond that seven day expiry period.

Of course, the above is not the final solution and such a workaround will work for some time. There is no guarantee that if tomorrow Apple release ITP 2.2 or 3.0, the mechanism will continue to work as it does now.
It is absolutely possible that the final version of the ITP is slightly different and the tests I have made are wrong. But one thing is certain – from now on, the reporting of consumer actions will be one little bit harder than it was before.

Yet, I am a big fan of the slightly anonymous browsing of the Internet. Once a month I clean up all websites data:

Additionally I have set Safari by default to open in private mode:

The same goes for my other iOS devices.

Safari ITP 2.1 DEMO Showing cookies expiring in 7 days

Интервю на Гласове със Андрей Райчев

Андрей описва миналото и прехода.

„попадат под най-страшното определение за човека, съществото с илюзии, формулирано от Лао Дзъ: Който не знае къде отива, отива другаде.
Те винаги, дори когато им се струва, че успяват, попадат другаде. Това е тяхна задължителна съдба“

Интервю с Андрей Райчев част 1
Интервю с Андрей Райчев част 2
Интервю с Андрей Райчев част 3

Интервю на Гласове със Андрей Райчев

Prince of Persia Cheats

Ако сте играли оригиналния Prince of Persia оттук:
https://archive.org/details/msdos_Prince_of_Persia_1990
ето и как да си улесните живота със малко чийтване.

Първо натискате Ctrl-Q за да излезете от играта. После го пускате така „prince megahit“. След което можете да натискате следните клавиши:
+ – добавя още време
Shift-L – минава на следващото ниво
Shift-T – дава още животи
Shift-S – лекува изгубени животи
Shift-W – позволява летене докато падате
K – убива противника на екрана

Има и още няколко комбинации които можете да видите тук

Prince of Persia Cheats

Редирект Хаос Мтел част 2

Както във предната част ви показах, че има проблеми при редиректи след по-детайлен анализ то се оказа, че те даже са и по-големи.

Намери се едно огледално копие на Debian:
http://debian.mobiltel.bg/

На следния адрес:
http://w4.mtel.net
може да бъде намерен сайта на GPS.BG:
http://www.gps.bg

Сайта на една известна адвокатска кантора може да бъде видян по няколко начина:
http://mbridge.mtel.net/
http://ipabg.mtel.net/
http://balkandji.mtel.net/
http://dom.mtel.net/
http://marisimone.mtel.net/
http://a.mtel.net/

Тук и
http://w6.mtel.net/
има някакъв изоставен Tomcat, който за сметка на това също е добре индексиран.

Далеч по-интересното е какво се случва със останалите домейни на придобити компании през времето. Това са 3 компании – Близу, СпектърНет и МегаЛан.

Започваме със Близу:
$ curl -I blizoo.bg
HTTP/1.1 301 Moved Permanently
Date: Fri, 17 Aug 2018 11:43:57 GMT
Server: Apache/2.4.25 (FreeBSD) OpenSSL/1.0.2k-freebsd PHP/7.0.17
Location: http://www.blizoo.bg/
Content-Type: text/html; charset=iso-8859-1

$ curl -I www.blizoo.bg
HTTP/1.1 301 Moved Permanently
Date: Fri, 17 Aug 2018 11:43:59 GMT
Server: Apache/2.4.25 (FreeBSD) OpenSSL/1.0.2k-freebsd PHP/7.0.17
X-Powered-By: PHP/7.0.17
Set-Cookie: PHPSESSID=vcpe31mj0lh1r66m3snkofrga4; path=/; domain=.blizoo.bg
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate
Pragma: no-cache
Location: https://www.a1.bg/blanding
Content-Type: text/html; charset=UTF-8

$ curl -I https://www.blizoo.bg
HTTP/1.1 301 Moved Permanently
Date: Fri, 17 Aug 2018 11:44:05 GMT
Server: Apache/2.4.25 (FreeBSD) OpenSSL/1.0.2k-freebsd PHP/7.0.17
X-Powered-By: PHP/7.0.17
Set-Cookie: PHPSESSID=esk78doapo8ncgda4j1ddffj75; path=/; domain=.blizoo.bg
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate
Pragma: no-cache
Location: https://www.a1.bg/blanding
Content-Type: text/html; charset=UTF-8

$ curl -I https://blizoo.bg
HTTP/1.1 301 Moved Permanently
Date: Fri, 17 Aug 2018 11:44:08 GMT
Server: Apache/2.4.25 (FreeBSD) OpenSSL/1.0.2k-freebsd PHP/7.0.17
Location: http://www.blizoo.bg/
Content-Type: text/html; charset=iso-8859-1

Както виждаме има някакви редиректи и те работят по някакъв начин. Интересното е, че стария сайт обаче още работи:
$ curl -I www.blizoo.bg/images/Root/downloads/gt/p3_pricelist_blizoo.pdf
HTTP/1.1 200 OK
Date: Fri, 17 Aug 2018 11:43:38 GMT
Server: Apache/2.4.25 (FreeBSD) OpenSSL/1.0.2k-freebsd PHP/7.0.17
Last-Modified: Thu, 28 Jan 2016 09:52:29 GMT
ETag: "118c7c-52a61e06d3140"
Accept-Ranges: bytes
Content-Length: 1150076
Cache-Control: max-age=86400
Expires: Sat, 18 Aug 2018 11:43:38 GMT
Content-Type: application/pdf

$ curl -I https://www.blizoo.bg/images/Root/downloads/gt/p3_pricelist_blizoo.pdf
HTTP/1.1 200 OK
Date: Fri, 17 Aug 2018 11:54:25 GMT
Server: Apache/2.4.25 (FreeBSD) OpenSSL/1.0.2k-freebsd PHP/7.0.17
Last-Modified: Thu, 28 Jan 2016 09:52:29 GMT
ETag: "118c7c-52a61e06d3140"
Accept-Ranges: bytes
Content-Length: 1150076
Cache-Control: max-age=86400
Expires: Sat, 18 Aug 2018 11:54:25 GMT
Content-Type: application/pdf

Или поне някакви части от него са все още видими. За да бъде объркването пълно си има и напълно работещи редиректи вътре в самия сайт:
$ curl -I blizoo.bg/images/Root/downloads/gt/p3_pricelist_blizoo.pdf
HTTP/1.1 301 Moved Permanently
Date: Fri, 17 Aug 2018 11:54:13 GMT
Server: Apache/2.4.25 (FreeBSD) OpenSSL/1.0.2k-freebsd PHP/7.0.17
Location: http://www.blizoo.bg/images/Root/downloads/gt/p3_pricelist_blizoo.pdf
Cache-Control: max-age=2592000
Expires: Sun, 16 Sep 2018 11:54:13 GMT
Content-Type: text/html; charset=iso-8859-1

$ curl -I https://blizoo.bg/images/Root/downloads/gt/p3_pricelist_blizoo.pdf
HTTP/1.1 301 Moved Permanently
Date: Fri, 17 Aug 2018 11:54:20 GMT
Server: Apache/2.4.25 (FreeBSD) OpenSSL/1.0.2k-freebsd PHP/7.0.17
Location: http://www.blizoo.bg/images/Root/downloads/gt/p3_pricelist_blizoo.pdf
Cache-Control: max-age=2592000
Expires: Sun, 16 Sep 2018 11:54:20 GMT
Content-Type: text/html; charset=iso-8859-1

И ето още една част:

$ curl -I https://www.blizoo.bg/images/Root/downloads/gt/
HTTP/1.1 403 Forbidden
Date: Fri, 17 Aug 2018 11:56:31 GMT
Server: Apache/2.4.25 (FreeBSD) OpenSSL/1.0.2k-freebsd PHP/7.0.17
Content-Type: text/html; charset=iso-8859-1

$ curl -I http://www.blizoo.bg/images/Root/downloads/gt/
HTTP/1.1 403 Forbidden
Date: Fri, 17 Aug 2018 11:56:45 GMT
Server: Apache/2.4.25 (FreeBSD) OpenSSL/1.0.2k-freebsd PHP/7.0.17
Content-Type: text/html; charset=iso-8859-1< $ curl -I https://blizoo.bg/images/Root/downloads/gt/
HTTP/1.1 301 Moved Permanently
Date: Fri, 17 Aug 2018 11:56:37 GMT
Server: Apache/2.4.25 (FreeBSD) OpenSSL/1.0.2k-freebsd PHP/7.0.17
Location: http://www.blizoo.bg/images/Root/downloads/gt/
Content-Type: text/html; charset=iso-8859-1

$ curl -I http://blizoo.bg/images/Root/downloads/gt/
HTTP/1.1 301 Moved Permanently
Date: Fri, 17 Aug 2018 11:56:42 GMT
Server: Apache/2.4.25 (FreeBSD) OpenSSL/1.0.2k-freebsd PHP/7.0.17
Location: http://www.blizoo.bg/images/Root/downloads/gt/
Content-Type: text/html; charset=iso-8859-1
/code>

И още един файл който е важен:
$ curl -I http://www.blizoo.bg/robots.txt
HTTP/1.1 200 OK
Date: Fri, 17 Aug 2018 12:02:53 GMT
Server: Apache/2.4.25 (FreeBSD) OpenSSL/1.0.2k-freebsd PHP/7.0.17
Last-Modified: Tue, 04 Dec 2012 15:52:37 GMT
ETag: "18-4d008dadbdb40"
Accept-Ranges: bytes
Content-Length: 24
Content-Type: text/plain

Това допълнително обърква роботите и те продължават да се опитват да индексират стария сайт на Близу.
Затова навсякъде се казва когато се прави миграция на домейн към друг да се прави цялостен 301 редирект от стария към новия.

Втората закупена компания е Мегалан:
curl -I megalan.bg
HTTP/1.1 302 Redirect
Content-Length: 145
Content-Type: text/html; charset=UTF-8
Location: https://www.megalan.bg
Server: Microsoft-IIS/8.5
Date: Fri, 17 Aug 2018 12:07:41 GMT

$ curl -I www.megalan.bg -L
curl: (7) Failed to connect to www.megalan.bg port 80: Operation timed out

$ curl -I https://www.megalan.bg -L
curl: (7) Failed to connect to www.megalan.bg port 443: Network is unreachable

$ curl -I megalan.bg/robots.txt -L
HTTP/1.1 302 Redirect
Content-Length: 145
Content-Type: text/html; charset=UTF-8
Location: https://www.megalan.bg
Server: Microsoft-IIS/8.5
Date: Fri, 17 Aug 2018 12:25:26 GMT

За жалост тук не виждаме никакъв реализиран редирект и не можем да осъществим връзка със www.megalan.bg.
Което по същество си е още една пропусната възможност защото домейна беше много добре индексиран във миналото като интернет доставчик за София.

И последния участник е Спектърнет. По стар народен обичай за тукашните ширини те са използвали 2 домейна като основни:
spnet.net
spectrumnet.bg
а година преди придобиването им от МТел бяха закупили и небезизвестните Орбител:
orbitel.bg
заедно със хитовия им проект тогава
hit.bg

Но нека да започнем по реда им:
$ curl spnet.net -I
HTTP/1.1 403 Forbidden
Date: Fri, 17 Aug 2018 12:37:57 GMT
Server: Apache
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
X-Robots-Tag: none
X-Download-Options: noopen
X-Permitted-Cross-Domain-Policies: none
Content-Type: text/html; charset=iso-8859-1

$ curl www.spnet.net -I
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Fri, 17 Aug 2018 12:38:00 GMT
Content-Type: text/html; charset=iso-8859-1
Connection: close
Location: http://www.mtel.bg/spnet
Vary: Accept-Encoding

$ curl https://spnet.net -I
HTTP/1.1 403 Forbidden
Date: Fri, 17 Aug 2018 12:39:04 GMT
Server: Apache
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
X-Robots-Tag: none
X-Download-Options: noopen
X-Permitted-Cross-Domain-Policies: none
Last-Modified: Thu, 16 Oct 2014 13:20:58 GMT
Accept-Ranges: bytes
Content-Length: 4897
Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform
Pragma: no-cache
Content-Type: text/html; charset=UTF-8

$ curl https://www.spnet.net -I -k
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Fri, 17 Aug 2018 12:39:11 GMT
Content-Type: text/html; charset=iso-8859-1
Connection: close
Location: http://www.mtel.bg/spnet
Vary: Accept-Encoding

Тук вече редиректите са реализирани, но само на ниво www. Когато се достъпва домейна без www се връща грешка 403.
Интересното е, че на този домейн е вързано и огледалното копие на Debian:
http://debian.spnet.net/
но то може да се достъпи и ето така:
http://bbgra.spnet.net

Има обаче проблем при достъпа на spectrumnet.bg:
$ curl -I http://www.spectrumnet.bg/
curl: (7) Failed to connect to www.spectrumnet.bg port 80: Operation timed out
$ curl -I http://spectrumnet.bg/
curl: (7) Failed to connect to spectrumnet.bg port 80: Operation timed out
$ curl -I https://www.spectrumnet.bg/
curl: (7) Failed to connect to www.spectrumnet.bg port 443: Operation timed out
$ curl -I https://spectrumnet.bg/
curl: (7) Failed to connect to spectrumnet.bg port 443: Operation timed out

Така, че този домейн също попада във графата "пропуснати възможности".

Нека да видим и какво се случва със Орбител:
$ curl -I orbitel.bg
curl: (6) Could not resolve host: orbitel.bg
$ curl -I www.orbitel.bg
curl: (6) Could not resolve host: www.orbitel.bg

И Хит:
$ curl -I hit.bg
curl: (7) Failed to connect to hit.bg port 80: Operation timed out
$ curl -I www.hit.bg
curl: (7) Failed to connect to www.hit.bg port 80: Operation timed out

Още една "пропусната възможност".

Накратко както виждате сами вместо да се направят хубави редиректи и да се усили силата основния домейн (вече) А1.
Редиректите където са направени или са половинчато направени като допълнително объркват ботовете или самите домейни не работят поради някаква причина.

Може би мениджърите на А1 отговорни за онлайн дейностите би било хубаво да направят един хубав одит на сайтовете си преди да продължат напред със развиването на основния си сайт.

Редирект Хаос Мтел част 2

Редирект Хаос Мтел

Сега ще ви покажа как неправилни редиректи може да ви навредят много.

Както знаете Мтел вече не съществува и имаме А1. Нека да видим какво се случва със редиректите на сайта им?

Започваме със https://mobiltel.bg

$ curl https://mobiltel.bg/ -L -I -k
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Thu, 16 Aug 2018 12:02:33 GMT
Content-Type: text/html; charset=iso-8859-1
Connection: close
Location: http://www.mtel.bg/
Vary: Accept-Encoding

HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Thu, 16 Aug 2018 12:02:33 GMT
Content-Type: text/html; charset=iso-8859-1
Connection: close
Location: http://www.a1.bg/mtel-stava-A1/
Vary: Accept-Encoding

HTTP/1.1 200 OK
Server: nginx
Date: Thu, 16 Aug 2018 12:02:33 GMT
Content-Type: text/html
Connection: close
Last-Modified: Tue, 07 Aug 2018 08:31:39 GMT
ETag: "181300-e8b2-572d43907d0c0"
Accept-Ranges: bytes
Content-Length: 59570
Vary: Accept-Encoding

И проблема лъсна – сайта на A1 може да се гледа и под HTTP. Продължаваме със http

$ curl http://mobiltel.bg/ -L -I -k
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Thu, 16 Aug 2018 12:03:32 GMT
Content-Type: text/html; charset=iso-8859-1
Connection: close
Location: http://www.mtel.bg/
Vary: Accept-Encoding

HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Thu, 16 Aug 2018 12:03:32 GMT
Content-Type: text/html; charset=iso-8859-1
Connection: close
Location: http://www.a1.bg/mtel-stava-A1/
Vary: Accept-Encoding

HTTP/1.1 200 OK
Server: nginx
Date: Thu, 16 Aug 2018 12:03:32 GMT
Content-Type: text/html
Connection: close
Last-Modified: Tue, 07 Aug 2018 08:31:39 GMT
ETag: "182509-e8b2-572d43907d0c0"
Accept-Ranges: bytes
Content-Length: 59570
Vary: Accept-Encoding

И добавяме и www със и без HTTPS
$ curl http://www.mobiltel.bg/ -L -I -k
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Thu, 16 Aug 2018 12:04:49 GMT
Content-Type: text/html; charset=iso-8859-1
Connection: close
Location: http://www.mtel.bg/
Vary: Accept-Encoding

HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Thu, 16 Aug 2018 12:04:49 GMT
Content-Type: text/html; charset=iso-8859-1
Connection: close
Location: http://www.a1.bg/mtel-stava-A1/
Vary: Accept-Encoding

HTTP/1.1 200 OK
Server: nginx
Date: Thu, 16 Aug 2018 12:04:50 GMT
Content-Type: text/html
Connection: close
Last-Modified: Tue, 07 Aug 2018 08:31:39 GMT
ETag: "181300-e8b2-572d43907d0c0"
Accept-Ranges: bytes
Content-Length: 59570
Vary: Accept-Encoding

$ curl https://www.mobiltel.bg/ -L -I -k
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Thu, 16 Aug 2018 12:04:55 GMT
Content-Type: text/html; charset=iso-8859-1
Connection: close
Location: http://www.mtel.bg/
Vary: Accept-Encoding

HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Thu, 16 Aug 2018 12:04:55 GMT
Content-Type: text/html; charset=iso-8859-1
Connection: close
Location: http://www.a1.bg/mtel-stava-A1/
Vary: Accept-Encoding

HTTP/1.1 200 OK
Server: nginx
Date: Thu, 16 Aug 2018 12:04:55 GMT
Content-Type: text/html
Connection: close
Last-Modified: Tue, 07 Aug 2018 08:31:39 GMT
ETag: "182509-e8b2-572d43907d0c0"
Accept-Ranges: bytes
Content-Length: 59570
Vary: Accept-Encoding

Интересното е, че редиректа работи само към основната страница. Ако се зададе вътрешна страница линка се предава към последващия редирект.

$ curl https://www.mobiltel.bg/robots.txt -L -I -k
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Thu, 16 Aug 2018 12:09:00 GMT
Content-Type: text/html; charset=iso-8859-1
Connection: close
Location: http://www.mtel.bg/robots.txt
Vary: Accept-Encoding

HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Thu, 16 Aug 2018 12:09:00 GMT
Content-Type: text/html; charset=iso-8859-1
Connection: close
Location: http://www.a1.bg/robots.txt
Vary: Accept-Encoding

HTTP/1.1 200 OK
Server: nginx
Date: Thu, 16 Aug 2018 12:09:00 GMT
Content-Type: text/plain
Connection: close
Last-Modified: Thu, 17 May 2018 12:29:47 GMT
ETag: "183f09-4b8-56c65fdaca4c0"
Accept-Ranges: bytes
Content-Length: 1208
Vary: Accept-Encoding

Продължаваме със един друг домейн mtel.net

$ curl http://www.mtel.net/ -L -I -k
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Thu, 16 Aug 2018 12:11:17 GMT
Content-Type: text/html; charset=iso-8859-1
Connection: close
Location: http://www.mtel.bg/
Vary: Accept-Encoding

HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Thu, 16 Aug 2018 12:11:17 GMT
Content-Type: text/html; charset=iso-8859-1
Connection: close
Location: http://www.a1.bg/mtel-stava-A1/
Vary: Accept-Encoding

HTTP/1.1 200 OK
Server: nginx
Date: Thu, 16 Aug 2018 12:11:17 GMT
Content-Type: text/html
Connection: close
Last-Modified: Tue, 07 Aug 2018 08:31:39 GMT
ETag: "182509-e8b2-572d43907d0c0"
Accept-Ranges: bytes
Content-Length: 59570
Vary: Accept-Encoding

$ curl http://mtel.net/ -L -I -k
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Thu, 16 Aug 2018 12:11:46 GMT
Content-Type: text/html; charset=iso-8859-1
Connection: close
Location: http://www.mtel.bg/
Vary: Accept-Encoding

HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Thu, 16 Aug 2018 12:11:46 GMT
Content-Type: text/html; charset=iso-8859-1
Connection: close
Location: http://www.a1.bg/mtel-stava-A1/
Vary: Accept-Encoding

HTTP/1.1 200 OK
Server: nginx
Date: Thu, 16 Aug 2018 12:11:46 GMT
Content-Type: text/html
Connection: close
Last-Modified: Tue, 07 Aug 2018 08:31:39 GMT
ETag: "182509-e8b2-572d43907d0c0"
Accept-Ranges: bytes
Content-Length: 59570
Vary: Accept-Encoding

Същия резултат е и под HTTPS със или без www отпред.

Следващия домейн е mtel.bg койо май издържа най-дълго:
$ curl http://www.mtel.bg/ -L -I -k
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Thu, 16 Aug 2018 12:21:34 GMT
Content-Type: text/html; charset=iso-8859-1
Connection: close
Location: http://www.a1.bg/mtel-stava-A1/
Vary: Accept-Encoding

HTTP/1.1 200 OK
Server: nginx
Date: Thu, 16 Aug 2018 12:21:34 GMT
Content-Type: text/html
Connection: close
Last-Modified: Tue, 07 Aug 2018 08:31:39 GMT
ETag: "182509-e8b2-572d43907d0c0"
Accept-Ranges: bytes
Content-Length: 59570
Vary: Accept-Encoding

$ curl http://mtel.bg/ -L -I -k
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Thu, 16 Aug 2018 12:21:41 GMT
Content-Type: text/html; charset=iso-8859-1
Connection: close
Location: http://www.a1.bg/mtel-stava-A1/
Vary: Accept-Encoding

HTTP/1.1 200 OK
Server: nginx
Date: Thu, 16 Aug 2018 12:21:41 GMT
Content-Type: text/html
Connection: close
Last-Modified: Tue, 07 Aug 2018 08:31:39 GMT
ETag: "246b22-e8b2-572d43907d0c0"
Accept-Ranges: bytes
Content-Length: 59570
Vary: Accept-Encoding

Същия резултат е и под HTTPS със или без WWW.

Доста по-интересно е тук:
http://www.a1.bg/sitemap.xml
където линковете са към:
http://www.mtel.bg/sitemap_www.xml
http://www.mtel.bg/sitemap_products.xml
и изглежда генерирането им е спряло някъде през Май 2018та.

Влизаме във сайтмапа със продуктите където също сочат към mtel.bg и си харесваме първия:
https://www.mtel.bg/ustroystva/laptopi/acer/travelmate-x349.3370
и отваряме сорса. Намираме още 2 линка към mtel.bg:
http://mtel.bg/public/file/id/ebd9dc98957b723ce7be035e56dfcfe9/acer-travelmate-x349-product.png
https://media.mtel.bg/newsite2015/images/mega-menu/myki.jpg

Втория домейн може да се зареди със или без HTTPS разбира се. По-фрапантното е, че същата снимка може да се намери и тук:
http://media.a1.bg/newsite2015/images/mega-menu/myki.jpg
https://media.a1.bg/newsite2015/images/mega-menu/myki.jpg

Разбира се този хаос има и една приятна новина – все пак логването минава задължително през HTTPS:
$ curl -I http://a1.bg/moyat-a1-upravlyavay-sam-svoite-uslugi -L
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Thu, 16 Aug 2018 12:31:39 GMT
Content-Type: text/html; charset=iso-8859-1
Connection: close
Location: http://www.a1.bg/moyat-a1-upravlyavay-sam-svoite-uslugi
Vary: Accept-Encoding

HTTP/1.1 302 Found
Server: nginx
Date: Thu, 16 Aug 2018 12:31:39 GMT
Content-Type: text/html
Connection: close
Set-Cookie: PHPSESSID=i7hoh0m33jdqkf36rldg41vlo6; path=/; domain=www.a1.bg; HttpOnly
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: PHPSESSID=dss9df9253jluc4jbejkpjtqa3; path=/; domain=www.a1.bg; HttpOnly
Set-Cookie: device=desktop; expires=Sun, 13-Aug-2028 12:31:39 GMT; path=/; domain=.a1.bg
Set-Cookie: device=desktop; expires=Sun, 13-Aug-2028 12:31:39 GMT; path=/; domain=.a1.bg
Set-Cookie: device=desktop; expires=Sun, 13-Aug-2028 12:31:39 GMT; path=/; domain=.a1.bg
Location: https://www.a1.bg/moyat-a1-upravlyavay-sam-svoite-uslugi
Vary: Accept-Encoding

HTTP/1.1 200 OK
Server: nginx
Date: Thu, 16 Aug 2018 12:31:41 GMT
Content-Type: text/html
Connection: close
Set-Cookie: PHPSESSID=kl0lfvrmpp4pa4182fm77mrfm3; path=/; domain=www.a1.bg; HttpOnly
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: PHPSESSID=v3a01obe7oocra81c2hceir6e1; path=/; domain=www.a1.bg; HttpOnly
Set-Cookie: device=desktop; expires=Sun, 13-Aug-2028 12:31:41 GMT; path=/; domain=.a1.bg
Set-Cookie: device=desktop; expires=Sun, 13-Aug-2028 12:31:41 GMT; path=/; domain=.a1.bg
Set-Cookie: device=desktop; expires=Sun, 13-Aug-2028 12:31:41 GMT; path=/; domain=.a1.bg
Vary: Accept-Encoding

$ curl -I http://www.a1.bg/moyat-a1-upravlyavay-sam-svoite-uslugi -L
HTTP/1.1 302 Found
Server: nginx
Date: Thu, 16 Aug 2018 12:32:04 GMT
Content-Type: text/html
Connection: close
Set-Cookie: PHPSESSID=q36aqoaea4udfln23p4p5mms00; path=/; domain=www.a1.bg; HttpOnly
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: PHPSESSID=t8nns92n1i1qjme3b55dgnoq03; path=/; domain=www.a1.bg; HttpOnly
Set-Cookie: device=desktop; expires=Sun, 13-Aug-2028 12:32:04 GMT; path=/; domain=.a1.bg
Set-Cookie: device=desktop; expires=Sun, 13-Aug-2028 12:32:04 GMT; path=/; domain=.a1.bg
Set-Cookie: device=desktop; expires=Sun, 13-Aug-2028 12:32:04 GMT; path=/; domain=.a1.bg
Location: https://www.a1.bg/moyat-a1-upravlyavay-sam-svoite-uslugi
Vary: Accept-Encoding

HTTP/1.1 200 OK
Server: nginx
Date: Thu, 16 Aug 2018 12:32:05 GMT
Content-Type: text/html
Connection: close
Set-Cookie: PHPSESSID=bl1fub9vcmqlnjmug87nsam4a7; path=/; domain=www.a1.bg; HttpOnly
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: PHPSESSID=di8qdhfdkd56rg68ca8757kta4; path=/; domain=www.a1.bg; HttpOnly
Set-Cookie: device=desktop; expires=Sun, 13-Aug-2028 12:32:04 GMT; path=/; domain=.a1.bg
Set-Cookie: device=desktop; expires=Sun, 13-Aug-2028 12:32:04 GMT; path=/; domain=.a1.bg
Set-Cookie: device=desktop; expires=Sun, 13-Aug-2028 12:32:04 GMT; path=/; domain=.a1.bg
Vary: Accept-Encoding

Обаче защо със 302 редирект?

И последен тест със новия домейн А1:

$ curl -I http://a1.bg/test -L
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Thu, 16 Aug 2018 12:37:21 GMT
Content-Type: text/html; charset=iso-8859-1
Connection: close
Location: http://www.a1.bg/test
Vary: Accept-Encoding

HTTP/1.1 404 Not Found
Server: nginx
Date: Thu, 16 Aug 2018 12:37:21 GMT
Content-Type: text/html
Connection: close
Set-Cookie: PHPSESSID=em2uptm6ja7735d74ons6srka1; path=/; domain=www.a1.bg; HttpOnly
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: PHPSESSID=bjfopn6pgd82glq69cp9sumt97; path=/; domain=www.a1.bg; HttpOnly
Set-Cookie: device=desktop; expires=Sun, 13-Aug-2028 12:37:21 GMT; path=/; domain=.a1.bg
Vary: Accept-Encoding

$ curl -I https://a1.bg/test -L
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Thu, 16 Aug 2018 12:36:41 GMT
Content-Type: text/html; charset=iso-8859-1
Connection: close
Location: http://www.a1.bg/test
Vary: Accept-Encoding

HTTP/1.1 404 Not Found
Server: nginx
Date: Thu, 16 Aug 2018 12:36:41 GMT
Content-Type: text/html
Connection: close
Set-Cookie: PHPSESSID=rurlb92cva9hkr3tac8mf95l81; path=/; domain=www.a1.bg; HttpOnly
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: PHPSESSID=u9190l2kanjvkbl1jajm2s3b33; path=/; domain=www.a1.bg; HttpOnly
Set-Cookie: device=desktop; expires=Sun, 13-Aug-2028 12:36:41 GMT; path=/; domain=.a1.bg
Vary: Accept-Encoding

Разбира се за да бъде хаоса пълен canonical на техните адреси сочи към HTTPS://WWW което допълнително обърква бота кой е основния домейн.

Накратко ако правите редиректи не правете като тях. Това е и основната причина сайта все още да се вижда така:
site:mtel.bg
site:mobiltel.bg
site:mtel.net
и новия A1 да е частично индексиран близо 3 месеца след пускането му.

Статията има допълнение което може да бъде прочетено ето тук.

Редирект Хаос Мтел

Прескачане на paywalls

Един от най-хитрите начини за прескачане на платените механизми на сайтовете са следните два кода
javascript:window.location="https://m.facebook.com/l.php?u="+encodeURIComponent(window.location.href);
или
javascript:window.location="https://www.google.com/url?rct=j&url="+encodeURIComponent(window.location.href);

Как работят?

Маркирате един от тях и го завлачвате във лентата за отметките. Когато някой сайт започне да мрънка как не сте си платили за да видите статията избирате отметката записана по рано и си я прочитате без проблем. Защото тези paywalls всъщност внимават когато посетител дойде от Facebook или Google и им показва цялата статия. Със помоща на тези 2 кода вие излъгвате сайта, че идвате  баш от тях.

Прескачане на paywalls

Как да смъкнем картинки от Google Images Search?

Ако някога ви се е налагало да смъквате много картинки от Google по зададена ключова дума трябва да сте били отегчени до смърт от цялото това кликане и записване. Наскоро и на мен ми се наложи и точно поради това намерих един скрипт и ще ви покажа как да го използвате.

var cont=document.getElementsByTagName("body")[0];
var imgs=document.getElementsByTagName("a");
var i=0;var divv= document.createElement("div");
var aray=new Array();var j=-1;
while(++i<imgs.length){

  if(imgs[i].href.indexOf("/imgres?imgurl=http")>0){
divv.appendChild(document.createElement("br"));
aray[++j]=decodeURIComponent(imgs[i].href).split(/=|%|&/)[1].split("?imgref")[0];
divv.appendChild(document.createTextNode(aray[j]));
}
}
cont.insertBefore(divv,cont.childNodes[0]);

Как  да го използваме?

  1. Отваряме любимия си браузър и започваме да търсим по картинка.
  2. Когато се заредят снимките натискаме нейде десен бутон на мишката -> Inspect Element и отиваме на таба „Console“
  3. Копираме  скрипта отгоре и го поставяме там. Натискаме Enter. На някой браузъри може първите няколко резултата да са подпъхнати под хедъра. Махаме хедъра който пречи.
  4. Намираме всички линкове, копираме ги във нов текстов файл. Запазваме текстовия файл примерно като list.txt
  5. Изпълняваме една от следните команди във команден ред:
    wget -i list.txt
    cat file-with-list-of-urls.txt | xargs -n 1 curl -LO
    cat file-with-list-of-urls.txt | xargs -n 1 -P 10 curl -LO
    .
    Резултата и от трите команди е, че файловете се смъкват локално на компютъра. Ако първите две смъкват файл по файл то последната смъква по 10 файла наведнъж.

Така процеса е полу-автоматизиран и много по-бърз.

Допълнение от Владимир Василев за което се бях сетил, но съм го пропуснал:
Внимавайте за лицензите! Това, че смъквате изображенията не означава, че имате права върху тях самите. Ако искате да ви бъдат показани само такива изображения със съотвените лицензи ползвайте инструментите на търсачката. Безразборното използване на чужди изображения без необходимите права върху тях може да ви докара правни проблеми със собственниците на правата.

google images лицензи

Затова ако ще използвате снимките някъде проверете лицензите преди това.

SaveSave

Как да смъкнем картинки от Google Images Search?