Pamyat Naroda indexes. 3,400,000+ records processed

Discussions on archives and similar issues. Hosted by John Calvin and Jeff Leach.
User avatar
Jeff Leach
Host - Archive section
Posts: 1154
Joined: 19 Jan 2010 09:08
Location: Stockholm, Sweden

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by Jeff Leach » 27 Jun 2018 19:34

oleg62 wrote:Hi! Update is-now spread IBD gunners ,including over 41
That is really difficult to understand Oleg. If your english is by tell me in Russian.

oleg62
Member
Posts: 31
Joined: 19 Feb 2010 18:16

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by oleg62 » 27 Jun 2018 19:42

Fill in the search for the word (dad ) ( gap) (cap) and you will all new files . There and look Fund, inventory, etc.

User avatar
Der Alte Fritz
Member
Posts: 1865
Joined: 13 Dec 2007 21:43
Location: Kent United Kingdom

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by Der Alte Fritz » 28 Jun 2018 06:32

I have a long list of book-marked material that I identified back in January that was only marked by placeholders. Some of this has now started to appear for instance this pre-war establishment table for a Mechanised Corps (https://pamyat-naroda.ru/documents/view/?id=136100934) but the bulk of it remains unfilled as yet. New files are being added as well because when I did a record count back in February there were 1,685,953 records and there are now 1,795,011 today

For the calendar year 1941 there are 332,264 records compared to 1944 or 1/5th and these proportions have remained constant with the new releases, the bulk of material is always slanted towards 1944-45 and this seems to represent the archives holdings rather than any conscious effort to avoid the early years.

«Никто не забыт, ничто не забыто»

User avatar
G. Trifkovic
Forum Staff
Posts: 2188
Joined: 06 Nov 2004 19:26
Location: The South-East

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by G. Trifkovic » 10 Aug 2018 23:02

Hi all.

https://vnr.github.io/pamyat-naroda-search/

is "temporarily" out of order; does anyone know what's going on and when (or if) will it be functional again?

Thanks,

G.

User avatar
Der Alte Fritz
Member
Posts: 1865
Joined: 13 Dec 2007 21:43
Location: Kent United Kingdom

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by Der Alte Fritz » 11 Aug 2018 06:41

A couple of months ago the IP address of the github/pamyat-naroda-search tool was blocked by the RF Ministry of Defence but this was easily avoided by using IP switching techniques.

However in the latest build of the website the Pamyat Naroda team have changed the way in which downloaded documents are displayed. Before when you clicked to download a document, it opened a new tab on your browser which showed the actual document image. This clearly displayed the actual address of the image itself. Today if you do the same thing, you will notice that this address is only displayed for an instant and then it will redirect to a page with a blank address line. What this is doing, is concealing the image addresses and it was these that search tools like github were using to catalogue the site and to produce those excellent precise lists that were so useful. It effect the TsAMO team have decided to make their website harder to hack but the effect of this is to make the life of researchers far harder, since their own search engine is so fuzzy in its logic that even a precise search brings up thousands of results.

Will it get fixed, I do not know but I am sure that someone will find a way to catalogue the website despite these protections.

User avatar
AMVAS
Member
Posts: 519
Joined: 02 Aug 2004 13:58
Location: Moscow

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by AMVAS » 11 Aug 2018 09:19

Der Alte Fritz wrote:
11 Aug 2018 06:41
A couple of months ago the IP address of the github/pamyat-naroda-search tool was blocked by the RF Ministry of Defence but this was easily avoided by using IP switching techniques.

However in the latest build of the website the Pamyat Naroda team have changed the way in which downloaded documents are displayed. Before when you clicked to download a document, it opened a new tab on your browser which showed the actual document image. This clearly displayed the actual address of the image itself. Today if you do the same thing, you will notice that this address is only displayed for an instant and then it will redirect to a page with a blank address line. What this is doing, is concealing the image addresses and it was these that search tools like github were using to catalogue the site and to produce those excellent precise lists that were so useful. It effect the TsAMO team have decided to make their website harder to hack but the effect of this is to make the life of researchers far harder, since their own search engine is so fuzzy in its logic that even a precise search brings up thousands of results.

Will it get fixed, I do not know but I am sure that someone will find a way to catalogue the website despite these protections.
Yes, in early July they did changed the method for access to their ElasticSearch engine. Before it was static link. Now it's dynamically generating links depending on time. I haven't study what they do with pages images loading, but as soon as they use the same Elastic Search engine for this it's also possible they made the same changes.
It's very annoying, because my graduating student has made alternative GUI for better usage of their search engine facilities including sorting, filtering and some other features, which are made in original site quite badly. But now, of course this work must be updated. Not sure now I have enough time to do this as soon as I need to develop new courses and do much other routine works
Last edited by AMVAS on 11 Aug 2018 11:11, edited 1 time in total.

User avatar
G. Trifkovic
Forum Staff
Posts: 2188
Joined: 06 Nov 2004 19:26
Location: The South-East

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by G. Trifkovic » 11 Aug 2018 10:08

Hi Der Alte Fritz and AMVAS,

and thanks for the info; please give us an update if anything changes.

Best,

G.

Mori
Member
Posts: 551
Joined: 25 Oct 2014 11:04

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by Mori » 11 Aug 2018 11:42

Unfortunately, it looks like another case example of the necessity to download whole websites while they are available / user-friendly, as this can change overnight...

User avatar
AMVAS
Member
Posts: 519
Joined: 02 Aug 2004 13:58
Location: Moscow

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by AMVAS » 11 Aug 2018 12:42

Mori wrote:
11 Aug 2018 11:42
Unfortunately, it looks like another case example of the necessity to download whole websites while they are available / user-friendly, as this can change overnight...
Indeed. I managed to download a large part of it, but my facilities are not infinite

User avatar
Der Alte Fritz
Member
Posts: 1865
Joined: 13 Dec 2007 21:43
Location: Kent United Kingdom

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by Der Alte Fritz » 11 Aug 2018 14:50

Looks like it has changed again as the dynamic link (ie. a blank page address) is no longer seen when downloading a file. Today it shows the normal image address as before which makes life a lot easier.
Screenshot 2018-08-11 14.39.35.png
Has anyone spoken to the designer of the github tool recently for an update?

When using the Pamyat Naroda search engine how do you find "new" recently added documents
You do not have the required permissions to view the files attached to this post.

User avatar
AMVAS
Member
Posts: 519
Joined: 02 Aug 2004 13:58
Location: Moscow

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by AMVAS » 11 Aug 2018 17:33

Der Alte Fritz wrote:
11 Aug 2018 14:50
Looks like it has changed again as the dynamic link (ie. a blank page address) is no longer seen when downloading a file. Today it shows the normal image address as before which makes life a lot easier.

Has anyone spoken to the designer of the github tool recently for an update?

When using the Pamyat Naroda search engine how do you find "new" recently added documents
Page links are still static. But search engine links are dynamic. See attach
You do not have the required permissions to view the files attached to this post.

User avatar
Der Alte Fritz
Member
Posts: 1865
Joined: 13 Dec 2007 21:43
Location: Kent United Kingdom

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by Der Alte Fritz » 12 Aug 2018 07:38

oleg62 wrote:
27 Jun 2018 19:42
Fill in the search for the word (dad ) ( gap) (cap) and you will all new files . There and look Fund, inventory, etc.
Jeff
Did we ever work out what Oleg was saying?

User avatar
AMVAS
Member
Posts: 519
Joined: 02 Aug 2004 13:58
Location: Moscow

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by AMVAS » 12 Aug 2018 12:33

Der Alte Fritz wrote:
12 Aug 2018 07:38
oleg62 wrote:
27 Jun 2018 19:42
Fill in the search for the word (dad ) ( gap) (cap) and you will all new files . There and look Fund, inventory, etc.
Jeff
Did we ever work out what Oleg was saying?
No ideas what does he meant under this

User avatar
Jeff Leach
Host - Archive section
Posts: 1154
Joined: 19 Jan 2010 09:08
Location: Stockholm, Sweden

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by Jeff Leach » 13 Aug 2018 05:24

Der Alte Fritz wrote:
12 Aug 2018 07:38
oleg62 wrote:
27 Jun 2018 19:42
Fill in the search for the word (dad ) ( gap) (cap) and you will all new files . There and look Fund, inventory, etc.
Jeff
Did we ever work out what Oleg was saying?
No, sometimes posts are too cryptic to figure out. In such cases, I usual ask them to post in their mother tongue but this didn't work in this case either.

Sean Oliver
Member
Posts: 55
Joined: 14 Sep 2007 18:18
Location: Wisconsin USA

Re: Pamyat Naroda indexes. 3,400,000+ records processed

Post by Sean Oliver » 17 Aug 2018 00:49

I find it hard to understand why Pamyat Naroda didn't simply group the docs into easily downloadable PDFs to begin with. This was an obviously important project for the Russian MoD, and it is in their presumed interest to make gathering the docs as easy as possible. What did they imagine people would DO with their material anyway?

Return to “Archives”