Re: MVS 3.8j (TK5) Crashes on Massive SORT
Hi Daniel,
The S/M Program runs in Problem Program mode (Key 8) and is not an Authorized Program as it does not use any system services that require Authorization. The Program strictly follows the published APIs for Supervisor services and Data Management services.
It does make extensive use of EXCP I/O to manage data flow to and from the Sort Work intermediate work data sets. When sufficient storage is available for double buffering of the Sort Work intermediate work data sets (as it would be in your case) then the S/M Program will drive the processor and the DASD for maximum utilization. You may have seen, with your sorting runs, that the processor time reported will be about 75% of the elapsed time demonstrating excellent overlap of processor and I/O processing.
I suspect that, with such a high degree of processor and I/O concurrency, there is a bug in MVS 3.8 when it is running in multiprocessor mode, possibly in the area of scheduling concurrent SRBs given the high I/O activity from just one address space.
Enabling multi processor mode in MVS 3.8 is not going to improve the performance of the S/M Program because it runs as a single task. The second processor would normally be idle, unless there was work available for another task, except when an I/O event completed and the second CPU was dispatched to run an SRB to post completion of the I/O event.
The fact that SYS1.LOGREC is filling up (with error recording ?) is an indicator to me that the problems is within MVS 3.8 internals.
IIRC, MVS 3.8 was not regarded as being particularly stable running in multi processor mode. It was the later PP extensions MVS/SE 1, MVS/SE 2 and MVS/SP that addressed many issues in this area.
Regards
Tom
|
Re: MVS 3.8j (TK5) Crashes on Massive SORT
Hi, Daniel,
?
You would have to SYSGEN MVS 3.8J for either an "AP" or "MP" configuration to get that to work.
?
You cannot just expect to throw additional CPUs at MVS 3.8J and expect it to work "out of the box" as-is.
?
Hope that helps,
?
Mark S. Waterbury
|
MVS 3.8j (TK5) Crashes on Massive SORT
I've been playing with a SORT job to stress test my system.? It sorts 99,999,999 random 80 byte records.? The job works just fine if I am running with 1 CPU.? Run MVS with 2 CPUs and it typically locks up, clocks, and sometimes I have to clear SYS1.LOGREC after an IPL.
?
As far as I have read, MVS 3.8j ought to work fine with 2 CPUs but I've verified this enough to know that I can reliably reproduce the issue.
?
If anyone has any ideas and would like me to produce some debug data to sift through, please let me know.? In the meantime, I'll run MVS with 1 CPU.
?
|
Re: SORT, REGION and MAIN
Hi Daniel,
For a successful sort the REGION size definitely has to be larger than the storage that the S/M Program is allocated by use of various S/M Program parameters. As to how much larger the REGION size has to be is dependent on how the S/M Program is invoked, JCL with or without user exits, or by another program. Assuming the simplest case of invoking the S/M Program via JCL and no user exits I would suggest that the REGION size should be at least 128K larger than the storage allocated to the S/M Program to avoid any out of storage issues.
The S/M Program has many configuration options and parameters provided to limit its storage usage. Constraining S/M Program storage usage is not so much of an issue in the Hercules environment however in a production environment it is an important consideration.
The Sort/Merge Program has poor locality of reference when running in a virtual storage system. This is because the Sort/Merge Program functions by moving records from input buffers to the Record Storage Area and then selects, using multiple compare instructions, the next record to be moved out to one of the output buffers. The impact of this logic is that the contents of the RSA and all the I/O buffers are referenced constantly during a sorting operation. Allocating 1024 KB of storage to the Sort/Merge Program will result in the need for a working set of at least 256 pages to avoid paging overhead. Allocating additional storage to the Sort/Merge Program above what is needed for optimum performance will result in an increased number of pages required for the working set.
The OS/360 Sort/Merge for MVS 3.8 Installation, Customization and Diagnosis Guide provides a detailed discussion on controlling S/M Program storage allocation and performance characteristics.
Regards
Tom
|
Re: SORT, REGION and MAIN
Can run with REGION=6M possibly larger. End of step message shows how much memory of 24, 31, 64 bit memory was actually used. On Tue, Nov 19, 2024 at 1:59?PM Daniel L. Srebnick via groups.io <dan@...> wrote:
How do the values of REGION JCL paramater and the SORT parameter MAIN relate to one another.
I suspect that REGION might need to be a bit larger than MAIN, but I cannot find any specific explanation.
I'm playing around with a SORT of 99,999,999 randomly generated records. To get things work, I set the REGION to 2048K and MAIN to 1984K. The SORT works, but I'd like to be able to come up with the required values more scientifically.
Running TK5/Update 3
-- Mike A Schwab, Springfield IL USA Where do Forest Rangers go to get away from it all?
|
How do the values of REGION JCL paramater and the SORT parameter MAIN relate to one another.
?
I suspect that REGION might need to be a bit larger than MAIN, but I cannot find any specific explanation.
?
I'm playing around with a SORT of 99,999,999 randomly generated records.? To get things work, I set the REGION to 2048K and MAIN to 1984K.? The SORT works, but I'd like to be able to come up with the required values more scientifically.
?
Running TK5/Update 3
|
Re: Understanding PDS in MVS 3.8
is a 3390 no key capacity table.? Goes all the way down to 22 byte records.
Gentlemen,
?
Thank you for your contributions, it looks like we finally found real answer.
?
I have found the last peace of it:
?
In this text we see that there is four gaps that accompany each member in PDS dataset.
Also, i presume, that the same gaps are present at in directory block.
As Silvio Losa pointed out, there is 1729 cells in 3390 track.
?
55996bytes / 1729cells = 32bytes in one cell.
?
Now lets calculate gaps for one member only. As i said earlier, there is going to be two occurrences of those gaps,
one for member and other for directory block(?).
?
15.75cells + 7cells + 7.75cells + 6.75cells = 37.25 or 38 cells
?
38cells * 32byte = 1216bytes(for one set of gaps)
?
But in PDS with only one member we have two sets.
?
1216 * 2 = 2432(which is almost exactly the number of free space left in our experiment)
?
Lets proof our observations with second example.
When there was 14 members in PDS, free space was equal to 231 lines.
?
231 lines * 80bytes = 18480 bytes of free space
?
For 14 members in PDS we should have 15 sets of gaps.
?
1216bytes * 15 = 18240
?
BINGO!!
?
Thanks to Harold Grovesteen we now know that in emulated CKD drives all those gaps just collected at the track end.
Hope you found this investigation entertaining! :)
?
Best wishes,
Andre
?
?
?
?
-- Mike A Schwab, Springfield IL USA
Where do Forest Rangers go to get away from it all?
|
Re: Understanding PDS in MVS 3.8
Sergio, thank you for your curiosity, and for bringing this question up here.
?
Here is a link to the book you asked for:
?
Best wishes,
Andre
|
Re: Understanding PDS in MVS 3.8
Thank you all :)
Since it was mentioned before, I knew that gaps had something to do, but I didn’t thought that all that “wasted” space was only because of gaps. And also, I didn’t know how to visualize/calculate the gaps.
Andre, where did you find that description of the gaps? I guess there is more interesting information where that text came from.
Kind Regards --
Sergio Lindo Mansilla
Software developer with DevOps?tendencies
Portfolio:?https://binary-sequence.github.io/
toggle quoted message
Show quoted text
El 18.11.2024, a las 19:36, Andre via groups.io <procritic@...> escribió:
Gentlemen,
?
Thank you for your contributions, it looks like we finally found real answer.
?
I have found the last peace of it:
<gaps.png>
?
In this text we see that there is four gaps that accompany each member in PDS dataset.
Also, i presume, that the same gaps are present at in directory block.
As Silvio Losa pointed out, there is 1729 cells in 3390 track.
?
55996bytes / 1729cells = 32bytes in one cell.
?
Now lets calculate gaps for one member only. As i said earlier, there is going to be two occurrences of those gaps,
one for member and other for directory block(?).
?
15.75cells + 7cells + 7.75cells + 6.75cells = 37.25 or 38 cells
?
38cells * 32byte = 1216bytes(for one set of gaps)
?
But in PDS with only one member we have two sets.
?
1216 * 2 = 2432(which is almost exactly the number of free space left in our experiment)
?
Lets proof our observations with second example.
When there was 14 members in PDS, free space was equal to 231 lines.
?
231 lines * 80bytes = 18480 bytes of free space
?
For 14 members in PDS we should have 15 sets of gaps.
?
1216bytes * 15 = 18240
?
BINGO!!
?
Thanks to Harold Grovesteen we now know that in emulated CKD drives all those gaps just collected at the track end.
Hope you found this investigation entertaining! :)
?
Best wishes,
Andre
?
?
?
?
|
Re: Understanding PDS in MVS 3.8
Gentlemen,
?
Thank you for your contributions, it looks like we finally found real answer.
?
I have found the last peace of it:
?
In this text we see that there is four gaps that accompany each member in PDS dataset.
Also, i presume, that the same gaps are present at in directory block.
As Silvio Losa pointed out, there is 1729 cells in 3390 track.
?
55996bytes / 1729cells = 32bytes in one cell.
?
Now lets calculate gaps for one member only. As i said earlier, there is going to be two occurrences of those gaps,
one for member and other for directory block(?).
?
15.75cells + 7cells + 7.75cells + 6.75cells = 37.25 or 38 cells
?
38cells * 32byte = 1216bytes(for one set of gaps)
?
But in PDS with only one member we have two sets.
?
1216 * 2 = 2432(which is almost exactly the number of free space left in our experiment)
?
Lets proof our observations with second example.
When there was 14 members in PDS, free space was equal to 231 lines.
?
231 lines * 80bytes = 18480 bytes of free space
?
For 14 members in PDS we should have 15 sets of gaps.
?
1216bytes * 15 = 18240
?
BINGO!!
?
Thanks to Harold Grovesteen we now know that in emulated CKD drives all those gaps just collected at the track end.
Hope you found this investigation entertaining! :)
?
Best wishes,
Andre
?
?
?
?
|
Re: Understanding PDS in MVS 3.8
Andre's response containing a hex editor's output was to your,
Sergio's, question regarding space usage with a PDS.
I have read with interest the various responses with regard to
actual hardware design and formatting on a physical hard drive.
Unfortunately, the hex editor of an emulated volume's content is
not the same as what is found on a hard drive.? There has been no
discussion of the format of an uncompressed hard drive volume as
emulated by Hercules, which is what I think is in use here.? This
topic is really a Hercules list question than an MVS list
question, but the context is in this list.
Having built for SATK ()
the ability to create an uncompressed volume I have some
familiarity with the format used by Hercules.
The maximum byte capacity of a track is used for each track of
the specified disk volume, in this case a 3390, if memory serves.
Hercules calculates the amount of bytes for the record being
written based upon actual track usage by the hardware.? So gaps
are accounted for.? This includes the track header data and R0
record.
However each record in the track including the count, key, and
data portions are written sequentially to the emulating file
WITHOUT any gaps.? And each record immediately follows its
predecessor.? All of the gaps then become "collected" together at
the end of each emulated track in the binary file used to emulate
the volume.
As I mentioned, this sort of discussion is more appropriate for
the Hercules list which is monitored by Fish.? The OS specific
lists get less attention from him.? Fish, if you see this message
please chime in with any clarifications, corrections, etc.
My real purpose is to help explain why you will see something
different in a hex editor of a Hercules CKD file than would occur
on a physical disk volume.
I hope this helps in understanding what you are observing.
Harold Grovesteen
On 11/17/24 04:52, Andre via groups.io
wrote:
toggle quoted message
Show quoted text
Please find attached png file. This is how it looks in hex
editor.
Why so many free space at the end?
|
Re: Understanding PDS in MVS 3.8
Dear all,
?
With a standard R0 on a track there remains 1729 user data cells on a 3390 track.
?
Number of cells needed is C + K + D
?
Where C = 10
?
K = 0? (if no key written)
?
DN = (DL + 6) / 232
D = 9 + (DL + (6 * DN) + 6) / 34) ???(where DL is the data length. Always rounded up to integer value)
?
?
So lets put in the values for the maximum size of 2 blocks:
?
(27998 + 6) / 232? = 121? (DN)
9 + (27998 + (6 * 121) + 6) / 34 = 854
?
Means space in cells =? C + K + D =? 10 + 0 + 854 = 864
So 2 blocks needs 2 * 864 cells = 1728?? (as initially said max. is 1729 cells per track with standard R0)
?
You can find the formulas in 3390 Reference Summary?? GX26-4577
?
Silvio
?
?
?
?
?
toggle quoted message
Show quoted text
From: [email protected] <[email protected]>
On Behalf Of Joe Monk via groups.io
Sent: Monday, 18 November 2024 14:16
To: [email protected]
Subject: Re: [H390-MVS] Understanding PDS in MVS 3.8
?
"The table you attached proves that you will only get 1 block per track at a block size of 28320. "
There's no Keys, so the K field is omitted.?
Looks to me like 2 of those fit in 56664 bytes.
Unless I am missing something (could very well be).
?
On Mon, Nov 18, 2024 at 5:52?AM laddiehanus via
<laddiehanus=[email protected]> wrote:
The table you attached proves that you will only get 1 block per track at a block size of 28320.?
It’s easy enough test before you post maybe?you should do that.
?
On Monday, November 18, 2024, 4:32 AM, Joe Monk via
<joemonk64=[email protected]> wrote:
Everyone,
Also, please remember that the CKD format on 3390 DASD is emulated. The actual format of the DASD is in cells, so there is some wastage trying to make?the CKD format work, with there being 1729 cells per track.
Second, they are using?a 27920 block size on a 3390. According to the book, that is only 98.8 percent space efficient.
By my calc, with a max blocksize?of 56664, divided by 80, yields 708 records per track, dividing for half-track blocking would give 354 records per half-track, for a max efficiency?blocksize of 28320.
So it seems to me to be working as designed.
?
On 4/11/2024 10:55 pm, Andre via wrote:
It is not that easy to see what Sergio is asking about, so i made a visualization.
When PDS has only one member with size of one track, oddly enough there is some space left at the end.
What its purpose remains unknown...
If, in the above picture, the white space is supposed to represent that part of the track that cannot be used to hold any accessible "thing" then the above picture is not accurate.
The white space should be distributed into several locations instead if being lumped into one area at the end.
Some places where I think the white space should be are:
-
the space between the track's record zero ("R0") and the directory block's key.
-
the space between the directory block's key and the directory block's data - which gives the electronics the time (as the disk rotates) to decide if the key value means that the block's data should be read or not.
-
the space between the the directory block and the EOF marker.
-
the space between the EOF marker and the member's first data block.
-
the space between the member's first data block and the member's second data block.
And in the points above I neglected the Count field of each physical record, so there are even more gaps that I describe.
You can use the LISTDS or LISTD command with the LABEL operand to show a data set's VTOC entry.
The DS1TRBAL field shows the unused byte balance for the last track of the data set containing data (for sequential and partitioned data sets).
The access method will look at the DS1TRBAL field to decide if a block of a given size will fit on that track or needs to go to the next track.
listd trk1 la?????????????????????????????????????????????????????????? ?
GREG.TRK1.CNTL????????????????????????????????????????????????????????? ?
--RECFM-LRECL-BLKSIZE-DSORG???????????????????????????????????????????? ?
? FB??? 80??? 27920?? PO??????????????????????????????????????????????? ?
--VOLUMES--???????????????????????????????????????????????????????????? ?
? TSO003??????????????????????????????????????????????????????????????? ?
--FORMAT 1 DSCB--?????????????????????????????????????????????????????? ?
F1 C3C2E3E4F0F3 0001 7C0143 000000 01 37 00 C9C2D4D6E2E5E2F24040404040? ?
7C014300000000 0200 90 00 6D10 0050 00 0000 82 80000000 000005 0044 0000
01000556000E0556000E 00000000000000000000 00000000000000000000 0000000000
So after performing the test procedure under discussion, the LISTD of the subject PDS is shown above.
As previously discussed, 668 records could not be saved on the first track, but 667 records could.
DS1LSTAR is 000005 as expected because 5 records were written to the track, those being:
-
The directory block.
-
The directory EOF.
-
The member's first block.
-
The member's second block.
-
The member's EOF.
DS1TRBAL is 0044 which equals 68 decimal.? You can't fit another 80-byte record in 68 bytes of space.
As previously mentioned, the gaps (of a sufficient angular size to provide sufficiently long (for the expected disk rotation speed) for electronics to react to the discovered situation) are essential in the CKD paradigm.? When CKD/ECKD DASD are emulated,
how the data is stored by the emulation scheme is arbitrary.
Cheers,
Greg
|
Re: Understanding PDS in MVS 3.8
Per IBM, the most efficient block size for a 3390 is 27,998
bytes, which allows two per track:
That is less than 28,320 bytes.
Interesting article which shows the calculations:
From that:
SPACE for a block = C + K + D
C = 10
K = 0
D = 9 + (Blocksize + (6 * DN) + 6)/34
DN = (Blocksize + 6)/232
BPT = 1729/SPACE??? (blocks per track)
Rounding up for D and DN
Thus for a block size of 28320 the math would be:
DN = 123
D = 864
SPACE = 874
BPT = 1.978 (in other words, one block will fit on a track, with
a lot of wasted space)
For 27998 as suggested by IBM:
DN= 121
D = 854
SPACE = 864
BPT = 2.00116 (so 2 can fit, with minimal wasted space)
On 11/18/24 08:16, Joe Monk via
groups.io wrote:
toggle quoted message
Show quoted text
"The table you attached proves that you will only
get 1 block per track at a block size of 28320. "
There's no Keys, so the K field is omitted.?
C = 5 bytes.
D = 28320 bytes.
Looks to me like 2 of those fit in 56664 bytes.
Unless I am missing something (could very well be).
Joe
On Mon, Nov 18, 2024 at
5:52?AM laddiehanus via <laddiehanus= [email protected]>
wrote:
The table you attached proves that you will only get 1 block
per track at a block size of 28320.?
It’s easy enough test before you post maybe?you should
do that.
On
Monday, November 18, 2024, 4:32 AM, Joe Monk via <joemonk64=[email protected]>
wrote:
Everyone,
Also, please remember that the CKD format on
3390 DASD is emulated. The actual format of the
DASD is in cells, so there is some wastage trying
to make?the CKD format work, with there being 1729
cells per track.
Second, they are using?a 27920 block size on a
3390. According to the book, that is only 98.8
percent space efficient.
By my calc, with a max blocksize?of 56664,
divided by 80, yields 708 records per track,
dividing for half-track blocking would give 354
records per half-track, for a max
efficiency?blocksize of 28320.
So it seems to me to be working as designed.
Joe
On 4/11/2024 10:55 pm, Andre via
wrote:
It is not that easy to see what Sergio is
asking about, so i made a visualization.
When PDS has only one member with size of
one track, oddly enough there is some space
left at the end.
What its purpose remains unknown...
If, in the above picture, the white space is
supposed to represent that part of the track
that cannot be used to hold any accessible
"thing" then the above picture is not accurate.
The white space should be distributed into
several locations instead if being lumped into
one area at the end.
Some places where I think the white space should
be are:
- the space between the track's record zero
("R0") and the directory block's key.
- the space between the directory block's
key and the directory block's data - which
gives the electronics the time (as the disk
rotates) to decide if the key value means
that the block's data should be read or not.
- the space between the the directory block
and the EOF marker.
- the space between the EOF marker and the
member's first data block.
- the space between the member's first data
block and the member's second data block.
And in the points above I neglected the Count
field of each physical record, so there are
even more gaps that I describe.
You can use the LISTDS or LISTD command with
the LABEL operand to show a data set's VTOC
entry.
The DS1TRBAL field shows the unused byte
balance for the last track of the data set
containing data (for sequential and
partitioned data sets).
The access method will look at the DS1TRBAL
field to decide if a block of a given size
will fit on that track or needs to go to the
next track.
listd trk1 la?????????????????????????????????????????????????????????? ?
GREG.TRK1.CNTL????????????????????????????????????????????????????????? ?
--RECFM-LRECL-BLKSIZE-DSORG???????????????????????????????????????????? ?
? FB??? 80??? 27920?? PO??????????????????????????????????????????????? ?
--VOLUMES--???????????????????????????????????????????????????????????? ?
? TSO003??????????????????????????????????????????????????????????????? ?
--FORMAT 1 DSCB--?????????????????????????????????????????????????????? ?
F1 C3C2E3E4F0F3 0001 7C0143 000000 01 37 00 C9C2D4D6E2E5E2F24040404040? ?
7C014300000000 0200 90 00 6D10 0050 00 0000 82 80000000 000005 0044 0000
01000556000E0556000E 00000000000000000000 00000000000000000000 0000000000
So after performing the test procedure under
discussion, the LISTD of the subject PDS is
shown above.
As previously discussed, 668 records could
not be saved on the first track, but 667
records could.
DS1LSTAR is 000005 as expected because 5
records were written to the track, those
being:
- The directory block.
- The directory EOF.
- The member's first block.
- The member's second block.
- The member's EOF.
DS1TRBAL is 0044 which equals 68 decimal.?
You can't fit another 80-byte record in 68
bytes of space.
As previously mentioned, the gaps (of a
sufficient angular size to provide
sufficiently long (for the expected disk
rotation speed) for electronics to react to
the discovered situation) are essential in the
CKD paradigm.? When CKD/ECKD DASD are
emulated, how the data is stored by the
emulation scheme is arbitrary.
Cheers,
Greg
|
Re: Understanding PDS in MVS 3.8
You can't have block that big.? You write two blocks with two sets of interecord gap.? And that table already assumes key length of 0.? There is a second table for key length 1-20.
"The table you attached proves that you will only get 1 block per track at a block size of 28320. "
There's no Keys, so the K field is omitted.?
C = 5 bytes. D = 28320 bytes.
Looks to me like 2 of those fit in 56664 bytes.
Unless I am missing something (could very well be).
Joe
On Mon, Nov 18, 2024 at 5:52?AM laddiehanus via <laddiehanus= [email protected]> wrote:
The table you attached proves that you will only get 1 block per track at a block size of 28320.?
It’s easy enough test before you post maybe?you should do that.
On Monday, November 18, 2024, 4:32 AM, Joe Monk via <joemonk64=[email protected]> wrote: Everyone,
Also, please remember that the CKD format on 3390 DASD is emulated. The actual format of the DASD is in cells, so there is some wastage trying to make?the CKD format work, with there being 1729 cells per track.
Second, they are using?a 27920 block size on a 3390. According to the book, that is only 98.8 percent space efficient.
By my calc, with a max blocksize?of 56664, divided by 80, yields 708 records per track, dividing for half-track blocking would give 354 records per half-track, for a max efficiency?blocksize of 28320.
So it seems to me to be working as designed.
Joe
On 4/11/2024 10:55 pm, Andre via
wrote:
It is not that easy to see what Sergio is asking about, so i
made a visualization.
When PDS has only one member with size of one track, oddly
enough there is some space left at the end.
What its purpose remains unknown...
If, in the above picture, the white space is supposed to represent
that part of the track that cannot be used to hold any accessible
"thing" then the above picture is not accurate.
The white space should be distributed into several locations instead
if being lumped into one area at the end.
Some places where I think the white space should be are:
- the space between the track's record zero ("R0") and the
directory block's key.
- the space between the directory block's key and the directory
block's data - which gives the electronics the time (as the disk
rotates) to decide if the key value means that the block's data
should be read or not.
- the space between the the directory block and the EOF marker.
- the space between the EOF marker and the member's first data
block.
- the space between the member's first data block and the
member's second data block.
And in the points above I neglected the Count field of each
physical record, so there are even more gaps that I describe.
You can use the LISTDS or LISTD command with the LABEL operand to
show a data set's VTOC entry.
The DS1TRBAL field shows the unused byte balance for the last
track of the data set containing data (for sequential and
partitioned data sets).
The access method will look at the DS1TRBAL field to decide if a
block of a given size will fit on that track or needs to go to the
next track.
listd trk1 la?????????????????????????????????????????????????????????? ?
GREG.TRK1.CNTL????????????????????????????????????????????????????????? ?
--RECFM-LRECL-BLKSIZE-DSORG???????????????????????????????????????????? ?
? FB??? 80??? 27920?? PO??????????????????????????????????????????????? ?
--VOLUMES--???????????????????????????????????????????????????????????? ?
? TSO003??????????????????????????????????????????????????????????????? ?
--FORMAT 1 DSCB--?????????????????????????????????????????????????????? ?
F1 C3C2E3E4F0F3 0001 7C0143 000000 01 37 00 C9C2D4D6E2E5E2F24040404040? ?
7C014300000000 0200 90 00 6D10 0050 00 0000 82 80000000 000005 0044 0000
01000556000E0556000E 00000000000000000000 00000000000000000000 0000000000
So after performing the test procedure under discussion, the
LISTD of the subject PDS is shown above.
As previously discussed, 668 records could not be saved on the
first track, but 667 records could.
DS1LSTAR is 000005 as expected because 5 records were written to
the track, those being:
- The directory block.
- The directory EOF.
- The member's first block.
- The member's second block.
- The member's EOF.
DS1TRBAL is 0044 which equals 68 decimal.? You can't fit another
80-byte record in 68 bytes of space.
As previously mentioned, the gaps (of a sufficient angular size
to provide sufficiently long (for the expected disk rotation
speed) for electronics to react to the discovered situation) are
essential in the CKD paradigm.? When CKD/ECKD DASD are emulated,
how the data is stored by the emulation scheme is arbitrary.
Cheers,
Greg
-- Mike A Schwab, Springfield IL USA
Where do Forest Rangers go to get away from it all?
|
Re: Understanding PDS in MVS 3.8
"The table you attached proves that you will only get 1 block per track at a block size of 28320. "
There's no Keys, so the K field is omitted.?
C = 5 bytes. D = 28320 bytes.
Looks to me like 2 of those fit in 56664 bytes.
Unless I am missing something (could very well be).
Joe
On Mon, Nov 18, 2024 at 5:52?AM laddiehanus via <laddiehanus= [email protected]> wrote:
The table you attached proves that you will only get 1 block per track at a block size of 28320.?
It’s easy enough test before you post maybe?you should do that.
On Monday, November 18, 2024, 4:32 AM, Joe Monk via <joemonk64=[email protected]> wrote: Everyone,
Also, please remember that the CKD format on 3390 DASD is emulated. The actual format of the DASD is in cells, so there is some wastage trying to make?the CKD format work, with there being 1729 cells per track.
Second, they are using?a 27920 block size on a 3390. According to the book, that is only 98.8 percent space efficient.
By my calc, with a max blocksize?of 56664, divided by 80, yields 708 records per track, dividing for half-track blocking would give 354 records per half-track, for a max efficiency?blocksize of 28320.
So it seems to me to be working as designed.
Joe
On 4/11/2024 10:55 pm, Andre via
wrote:
It is not that easy to see what Sergio is asking about, so i
made a visualization.
When PDS has only one member with size of one track, oddly
enough there is some space left at the end.
What its purpose remains unknown...
If, in the above picture, the white space is supposed to represent
that part of the track that cannot be used to hold any accessible
"thing" then the above picture is not accurate.
The white space should be distributed into several locations instead
if being lumped into one area at the end.
Some places where I think the white space should be are:
- the space between the track's record zero ("R0") and the
directory block's key.
- the space between the directory block's key and the directory
block's data - which gives the electronics the time (as the disk
rotates) to decide if the key value means that the block's data
should be read or not.
- the space between the the directory block and the EOF marker.
- the space between the EOF marker and the member's first data
block.
- the space between the member's first data block and the
member's second data block.
And in the points above I neglected the Count field of each
physical record, so there are even more gaps that I describe.
You can use the LISTDS or LISTD command with the LABEL operand to
show a data set's VTOC entry.
The DS1TRBAL field shows the unused byte balance for the last
track of the data set containing data (for sequential and
partitioned data sets).
The access method will look at the DS1TRBAL field to decide if a
block of a given size will fit on that track or needs to go to the
next track.
listd trk1 la?????????????????????????????????????????????????????????? ?
GREG.TRK1.CNTL????????????????????????????????????????????????????????? ?
--RECFM-LRECL-BLKSIZE-DSORG???????????????????????????????????????????? ?
? FB??? 80??? 27920?? PO??????????????????????????????????????????????? ?
--VOLUMES--???????????????????????????????????????????????????????????? ?
? TSO003??????????????????????????????????????????????????????????????? ?
--FORMAT 1 DSCB--?????????????????????????????????????????????????????? ?
F1 C3C2E3E4F0F3 0001 7C0143 000000 01 37 00 C9C2D4D6E2E5E2F24040404040? ?
7C014300000000 0200 90 00 6D10 0050 00 0000 82 80000000 000005 0044 0000
01000556000E0556000E 00000000000000000000 00000000000000000000 0000000000
So after performing the test procedure under discussion, the
LISTD of the subject PDS is shown above.
As previously discussed, 668 records could not be saved on the
first track, but 667 records could.
DS1LSTAR is 000005 as expected because 5 records were written to
the track, those being:
- The directory block.
- The directory EOF.
- The member's first block.
- The member's second block.
- The member's EOF.
DS1TRBAL is 0044 which equals 68 decimal.? You can't fit another
80-byte record in 68 bytes of space.
As previously mentioned, the gaps (of a sufficient angular size
to provide sufficiently long (for the expected disk rotation
speed) for electronics to react to the discovered situation) are
essential in the CKD paradigm.? When CKD/ECKD DASD are emulated,
how the data is stored by the emulation scheme is arbitrary.
Cheers,
Greg
|
Re: Understanding PDS in MVS 3.8
The table you attached proves that you will only get 1 block per track at a block size of 28320.?
It’s easy enough test before you post maybe?you should do that.
On Monday, November 18, 2024, 4:32 AM, Joe Monk via groups.io <joemonk64@...> wrote: Everyone,
Also, please remember that the CKD format on 3390 DASD is emulated. The actual format of the DASD is in cells, so there is some wastage trying to make?the CKD format work, with there being 1729 cells per track.
Second, they are using?a 27920 block size on a 3390. According to the book, that is only 98.8 percent space efficient.
By my calc, with a max blocksize?of 56664, divided by 80, yields 708 records per track, dividing for half-track blocking would give 354 records per half-track, for a max efficiency?blocksize of 28320.
So it seems to me to be working as designed.
Joe
On 4/11/2024 10:55 pm, Andre via
wrote:
It is not that easy to see what Sergio is asking about, so i
made a visualization.
When PDS has only one member with size of one track, oddly
enough there is some space left at the end.
What its purpose remains unknown...
If, in the above picture, the white space is supposed to represent
that part of the track that cannot be used to hold any accessible
"thing" then the above picture is not accurate.
The white space should be distributed into several locations instead
if being lumped into one area at the end.
Some places where I think the white space should be are:
- the space between the track's record zero ("R0") and the
directory block's key.
- the space between the directory block's key and the directory
block's data - which gives the electronics the time (as the disk
rotates) to decide if the key value means that the block's data
should be read or not.
- the space between the the directory block and the EOF marker.
- the space between the EOF marker and the member's first data
block.
- the space between the member's first data block and the
member's second data block.
And in the points above I neglected the Count field of each
physical record, so there are even more gaps that I describe.
You can use the LISTDS or LISTD command with the LABEL operand to
show a data set's VTOC entry.
The DS1TRBAL field shows the unused byte balance for the last
track of the data set containing data (for sequential and
partitioned data sets).
The access method will look at the DS1TRBAL field to decide if a
block of a given size will fit on that track or needs to go to the
next track.
listd trk1 la?????????????????????????????????????????????????????????? ?
GREG.TRK1.CNTL????????????????????????????????????????????????????????? ?
--RECFM-LRECL-BLKSIZE-DSORG???????????????????????????????????????????? ?
? FB??? 80??? 27920?? PO??????????????????????????????????????????????? ?
--VOLUMES--???????????????????????????????????????????????????????????? ?
? TSO003??????????????????????????????????????????????????????????????? ?
--FORMAT 1 DSCB--?????????????????????????????????????????????????????? ?
F1 C3C2E3E4F0F3 0001 7C0143 000000 01 37 00 C9C2D4D6E2E5E2F24040404040? ?
7C014300000000 0200 90 00 6D10 0050 00 0000 82 80000000 000005 0044 0000
01000556000E0556000E 00000000000000000000 00000000000000000000 0000000000
So after performing the test procedure under discussion, the
LISTD of the subject PDS is shown above.
As previously discussed, 668 records could not be saved on the
first track, but 667 records could.
DS1LSTAR is 000005 as expected because 5 records were written to
the track, those being:
- The directory block.
- The directory EOF.
- The member's first block.
- The member's second block.
- The member's EOF.
DS1TRBAL is 0044 which equals 68 decimal.? You can't fit another
80-byte record in 68 bytes of space.
As previously mentioned, the gaps (of a sufficient angular size
to provide sufficiently long (for the expected disk rotation
speed) for electronics to react to the discovered situation) are
essential in the CKD paradigm.? When CKD/ECKD DASD are emulated,
how the data is stored by the emulation scheme is arbitrary.
Cheers,
Greg
|
Re: Understanding PDS in MVS 3.8
Everyone,
Also, please remember that the CKD format on 3390 DASD is emulated. The actual format of the DASD is in cells, so there is some wastage trying to make?the CKD format work, with there being 1729 cells per track.
Second, they are using?a 27920 block size on a 3390. According to the book, that is only 98.8 percent space efficient.
By my calc, with a max blocksize?of 56664, divided by 80, yields 708 records per track, dividing for half-track blocking would give 354 records per half-track, for a max efficiency?blocksize of 28320.
So it seems to me to be working as designed.
Joe
On 4/11/2024 10:55 pm, Andre via
wrote:
It is not that easy to see what Sergio is asking about, so i
made a visualization.
When PDS has only one member with size of one track, oddly
enough there is some space left at the end.
What its purpose remains unknown...
If, in the above picture, the white space is supposed to represent
that part of the track that cannot be used to hold any accessible
"thing" then the above picture is not accurate.
The white space should be distributed into several locations instead
if being lumped into one area at the end.
Some places where I think the white space should be are:
- the space between the track's record zero ("R0") and the
directory block's key.
- the space between the directory block's key and the directory
block's data - which gives the electronics the time (as the disk
rotates) to decide if the key value means that the block's data
should be read or not.
- the space between the the directory block and the EOF marker.
- the space between the EOF marker and the member's first data
block.
- the space between the member's first data block and the
member's second data block.
And in the points above I neglected the Count field of each
physical record, so there are even more gaps that I describe.
You can use the LISTDS or LISTD command with the LABEL operand to
show a data set's VTOC entry.
The DS1TRBAL field shows the unused byte balance for the last
track of the data set containing data (for sequential and
partitioned data sets).
The access method will look at the DS1TRBAL field to decide if a
block of a given size will fit on that track or needs to go to the
next track.
listd trk1 la?????????????????????????????????????????????????????????? ?
GREG.TRK1.CNTL????????????????????????????????????????????????????????? ?
--RECFM-LRECL-BLKSIZE-DSORG???????????????????????????????????????????? ?
? FB??? 80??? 27920?? PO??????????????????????????????????????????????? ?
--VOLUMES--???????????????????????????????????????????????????????????? ?
? TSO003??????????????????????????????????????????????????????????????? ?
--FORMAT 1 DSCB--?????????????????????????????????????????????????????? ?
F1 C3C2E3E4F0F3 0001 7C0143 000000 01 37 00 C9C2D4D6E2E5E2F24040404040? ?
7C014300000000 0200 90 00 6D10 0050 00 0000 82 80000000 000005 0044 0000
01000556000E0556000E 00000000000000000000 00000000000000000000 0000000000
So after performing the test procedure under discussion, the
LISTD of the subject PDS is shown above.
As previously discussed, 668 records could not be saved on the
first track, but 667 records could.
DS1LSTAR is 000005 as expected because 5 records were written to
the track, those being:
- The directory block.
- The directory EOF.
- The member's first block.
- The member's second block.
- The member's EOF.
DS1TRBAL is 0044 which equals 68 decimal.? You can't fit another
80-byte record in 68 bytes of space.
As previously mentioned, the gaps (of a sufficient angular size
to provide sufficiently long (for the expected disk rotation
speed) for electronics to react to the discovered situation) are
essential in the CKD paradigm.? When CKD/ECKD DASD are emulated,
how the data is stored by the emulation scheme is arbitrary.
Cheers,
Greg
|
Re: Understanding PDS in MVS 3.8
On 4/11/2024 10:55 pm, Andre via
groups.io wrote:
It is not that easy to see what Sergio is asking about, so i
made a visualization.
When PDS has only one member with size of one track, oddly
enough there is some space left at the end.
What its purpose remains unknown...
If, in the above picture, the white space is supposed to represent
that part of the track that cannot be used to hold any accessible
"thing" then the above picture is not accurate.
The white space should be distributed into several locations instead
if being lumped into one area at the end.
Some places where I think the white space should be are:
- the space between the track's record zero ("R0") and the
directory block's key.
- the space between the directory block's key and the directory
block's data - which gives the electronics the time (as the disk
rotates) to decide if the key value means that the block's data
should be read or not.
- the space between the the directory block and the EOF marker.
- the space between the EOF marker and the member's first data
block.
- the space between the member's first data block and the
member's second data block.
And in the points above I neglected the Count field of each
physical record, so there are even more gaps that I describe.
You can use the LISTDS or LISTD command with the LABEL operand to
show a data set's VTOC entry.
The DS1TRBAL field shows the unused byte balance for the last
track of the data set containing data (for sequential and
partitioned data sets).
The access method will look at the DS1TRBAL field to decide if a
block of a given size will fit on that track or needs to go to the
next track.
listd trk1 la?????????????????????????????????????????????????????????? ?
GREG.TRK1.CNTL????????????????????????????????????????????????????????? ?
--RECFM-LRECL-BLKSIZE-DSORG???????????????????????????????????????????? ?
? FB??? 80??? 27920?? PO??????????????????????????????????????????????? ?
--VOLUMES--???????????????????????????????????????????????????????????? ?
? TSO003??????????????????????????????????????????????????????????????? ?
--FORMAT 1 DSCB--?????????????????????????????????????????????????????? ?
F1 C3C2E3E4F0F3 0001 7C0143 000000 01 37 00 C9C2D4D6E2E5E2F24040404040? ?
7C014300000000 0200 90 00 6D10 0050 00 0000 82 80000000 000005 0044 0000
01000556000E0556000E 00000000000000000000 00000000000000000000 0000000000
So after performing the test procedure under discussion, the
LISTD of the subject PDS is shown above.
As previously discussed, 668 records could not be saved on the
first track, but 667 records could.
DS1LSTAR is 000005 as expected because 5 records were written to
the track, those being:
- The directory block.
- The directory EOF.
- The member's first block.
- The member's second block.
- The member's EOF.
DS1TRBAL is 0044 which equals 68 decimal.? You can't fit another
80-byte record in 68 bytes of space.
As previously mentioned, the gaps (of a sufficient angular size
to provide sufficiently long (for the expected disk rotation
speed) for electronics to react to the discovered situation) are
essential in the CKD paradigm.? When CKD/ECKD DASD are emulated,
how the data is stored by the emulation scheme is arbitrary.
Cheers,
Greg
|
Re: Understanding PDS in MVS 3.8
Andre,
?
Despite what you may think, there are gaps everywhere on the DASD surface.? I dont know what hex dump you are looking at, but it isn't going to show the gaps.? If you are looking at a hex dump generated by Hercules, that is dumped from an emulated DASD.? But be assured that if a real DASD device, such as an actual 3350 or a 3380 device were being used, there would be gaps on every track.
?
There are many areas on each track that are set aside and are part of the overhead of managing the device.? On each and every track that is properly formatted, there is a Home Address marker, followed by a gap.? Then there is the count field for what is called "Record 0".? All count fields are eight bytes in size, and all of them are followed by a gap.? After the count field comes the key area.? The key is optional, so if it is not present then the data can follow immediately after the gap after the count.? If the key is present, it can be 1-255 bytes and is followed by a gap before the actual data portion.?
?
Record 0 is a special record managed by the operating system.? In most cases you cannot read or write Record 0 and it is not available to you.? After the count/key/data areas of Record 0, then there is a gap before the count field for the next record.? Followed by a gap, then the key and a gap, then the data. There is a then a gap before the next count field for the next record.? Rinse and repeat until no more complete count/key/data areas will fit on the track.
?
Note that the terminology when discussing DASD device architecture uses the term "record" to represent the count/key/data area, but in MVS and with the access methods, the data portion is called a block.
?
For a PDS, the first directory block will be after the gap after Record 0 on the track.? For the first directory block, and all subsequent directory blocks, there will be a count field, a gap, a key field, a gap, and the 256 data area of that directory block.? After the last directory block, there will be another count field with a data record length of 0; this is the EOF record.? It also has gaps between its count/key/data areas.? It doesnt count as a block in the total number of directory blocks. If you allocate 10 directory blocks you get ten sets of count/key/data with gaps all in between all of them, followed by the count/key of the EOF block.? The EOF block is the 11th block but isnt counted in your allocation or in any report of how many directory blocks there are.
?
It might seem wasteful to write an EOF record with its count and gap overhead.? But there is a reason.? When the hardware reads a count field with a data length of 0, it causes a special notice on the channel that the DASD is connected to called a 'unit exception'.? This exception is noted by the operating system and that is how it detects that the end of a file, or the end of the directory blocks, has been reached.
?
After the directory portion and when the data portion of the PDS dataset starts, it usually doesnt contain a key field, so typically there is a count, gap, data,gap.? For each block of data.? At the end of the member, there will be a block of zero length; the EOF block.? A new member may or may not follow after the gap after the EOF block.
?
Just because you can't see the gaps doesn't mean they arent there.? You can infer their presence in many cases.? Have you ever seen the DASD table that shows how many blocks of a certain size can fit on a track?? For example, on a 3390 you can fit 12 blocks of 4096 bytes on a track.? That is 4096*12=49152 bytes.? But the total track size of a 3390 is 56,664 bytes.? The lost space is due to all of the gaps between the count fields of each block, and the blocks themselves, and the overhead of the count/key/data and gaps of Record 0.
?
The smaller the block size, the worse it gets. This is because each block has the full overhead of a count/gap/data/gap.? I dont have the exact number of 1-byte blocks that can be written to a 3390 track in front of me, but it is something around 100 of them.?? 100 bytes of data on a 56,664-byte track!? That is awfully wasteful.? And that is why large blocks are recommended with multiple data records in each block.? Even two blocks of 27,920 bytes wastes space:? 27920*2=55,840.? So you are still wasting 824 bytes.? But it is the best we can do. MVS wont let us write more than 32760 bytes in a block.? So no matter how you slice it up, the best you can do is two blocks per track and bear the overhead of the resulting gaps.
?
Regards,
Bob
?
?
?
|
Re: Understanding PDS in MVS 3.8
The first track would contain the Directory block, the EOF Directory block, and the rest with PDS data blocks. On Sun, Nov 17, 2024 at 3:44?PM Andre via groups.io <procritic@...> wrote:
Well OK, then please provide your calculations concerning this problem.
Why there is free space at the end of PDS dataset that takes more than 2480 bytes?
You have said: " A pds has 256 byte blocks with a gap between each block. Then there is one block that makes end of file of the directory."
There is nothing more far from the truth, you can clearly see in hex dump of DASD that there is no 256 blocks apart from directory one,
No gaps at all. The PDS takes two blocks 27920 bytes each(1 track). Those blocks can have 349 lines of text each,
and this text stored on DASD as is, no gaps included. The only gap is between two blocks and it is 8 bytes.
Andre
On Mon, Nov 18, 2024 at 12:26 AM, laddiehanus wrote:
I have mislead no one. I have been working with this since 1982. There is overhead that can’t be seen with the tools that are publicly available.
Laddie Hanus
-- Mike A Schwab, Springfield IL USA Where do Forest Rangers go to get away from it all?
|
Silvio Losa